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The Relationship Between Hint mum Competency Testing Programs and 
Students 4 Reedfng Proficiency: Implications from the 1983-84 Notional 
Assessment of Educational Progress In Reeding end Writing 

Abstract 

Exploratory studies presented In tMs report wed multilevel data obtained In the 
1983-84 National Assessment of Educational Progress to Invratigata the relationship 
between minimum competency testing (MOT) programs and student reading proficiency. A 
"school effects* paradigm was used In Study 1 to assess the effect of a sohooHevel MOT 
program after adjusting for students' age, sex^egion of the wintry, family background, 
school- level composition, and socioeconomic status, 3^*013' academic behaviors, 
school- level remedial program, and instructional dollars per pupil. Within-reoe/ethnic 
group analyses were conducted to investigate whether or rot the effect of MOT programs on 
reading proficiency was similar for each race/ethnic group. The second study examined the 
relationship between the type of state testing program and reading proficiency for each graft 
cohort after adjusting for students 8 age, sex, district and school- level SES variables, family 
background, and race. These studies demoratrste the limits and potential of using NAEPdeta 
to Inform public policy issues* 
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Executive Summary 

Since ths mid-seventies, over 35 states have required local school districts to give 
minimum competency tests to students In elementary, junior high, and senior high school. 
In 1 984, forty states ware actively involved in some aspect of minimum competency testing 
(MOT); nineteen states were using test peformance as a basis for high school graduation, and 
five states were using tests as a basis for grade promotion. The legislation for MOT can be 
viewed within the larger context of the accountability movement in education which focuses 
concern upon the output of the educational system. MOT programs were designed to address 
accountability Issues such as the basic skills level of high school graduates, and to identity 
and provide remediation to students et other grade levels who failed to acquire basic skills. 
Many of these programs were initiated from outside the educational establishment and 
originated from legislators end state boards. However , schooi districts and schools have been 
required to implement these programs in growing numbers. 

Minimum comptency testing may be defined as a program to test students in terms of, 
and only in terms of, whatever competencies state or local authorities have decided are the 
minimally acceptable outcomes of an education. As a result, minimum competency is whet 
minimum competency tests assess. There are few commonly agreed upon definitions of the 
exact nature of minimum competency testing programs. Ststes differ dramatically on a 
number of dimensions, for example, economic climate, expenditures on education, and 
educational policies. As might be expected, they also differ in what they label and define as a 
minimum cwnpetency testing program. MOT programs vsry substantively as well as 
procedurally. Substantive differences Include the purposs, content, and grade levels 
assessed. Procedural differences Include the agency that sets the performance standards, the 
determination of passing scores, and the degree of discretion allowed local school districts. 
Of relevance to the studies included here was the content dimension, end the degree of 
discretion allowed local school districts. 
Content of MCT 

There are wide discrepancies in skills tested in minimum competency testing 
programs. The majority of states assess reading and methemeties; however, there is little 
raraerisus regarolng the assessment of survival or life skills. Assuming that the majority of 
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state mandated programs assess basic skills, the difficulty level of the content of MCT 
programs varies from state to state. Additionally, the various formulas used to drive 
passing scores limits the generalization and the evaluation of student outcomes across various 
types of programs, The differences between programs In each state make it virtually 
Impossible to assess the Impact of MCT on other than an intra-stete basis. 
Discretion Available to Lu ml Districts 

Moreover, differences between state MOT programs involve differing requirements, 
options, regulations, and guidelines to local school districts In implementing such programs. 
These differences result in varying degrees of discretion available to local school districts. 
For example, in some state programs, the purpose, the performance levels, the test, and the 
grade levels assessed, are determined at the state level; e.g. New Jersey. In other state 
programs, the actual test, the performance levels, grade levels assessed are left up to the 
local district. For example, in California, the state requires students to pass a test to 
graduate but allows the local districts to choose both the test and the passing score. 

The level of Implementation of a suite MCT program in a particular school may vary 
as a function of the amount of Ion) discretion allowed. Change can be successfully 
Implemented at the individual school level given the appropriate conditions, prooedurM, and 
support systems. Some research suggssts that the individual school is a unit of decision 
making with its own incentive structures. Thus, in tightly-controlled state MCT programs, 
principals and teachers who are actually responsible for students' learning am/ have a lower 
degree of ownership and less involvement in implementing state- mandated testing programs. 
In these particular schools, the effect of MCT programs on students' learning outcomes may 
differ from that found in schools functioning with more discretion; e,g,» selecting the test, 
grade level , or whether or not to Implement the state program. Schools that are successful 
In facilitating basic skills in reeding have been characterized by such school- level factors 
as, principals• instructional leadership, monitoring of student and teacher progress, high 
teacher expectations for students' achievement, curriculum articulation and organization, 
and school-wide staff development. Thus, we Important dimension of MCT programs 
considred in assessing student outcomes was the degree of discretion allowed to the local 
districts and schools. 
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Among the many unresolved issues surrounding MOT are, for ©cample, the effect on the 
curriculum, teachers and students' expectations, and the achievement of low-socioeconomic 
and minority students, Todate, information regarding achievement outcomes resulting from 
MOT programs derives primarily from intranstate assessment data. There is little or no 
information regarding the extent to which MOT programs implemented have led to 
improvement in student reeding achievement end higher educational standards in the nation. 
Investigations of these issues retire an objective measure of student reading achievement 
common to all state. Data collected in the 1983-84 National Assessment of Educational 
Progress ( NAEP) were used to provide some insight on these issues, 

NAEP is an on-going congressional ly- mandated project established to report on the 
educational achievement of American students. Since 1969, when NAEP was established, 
national assessments were conducted annually up until 1980. Fran 1980 to the present, 
assessments have been conducted biennially. The 1983-84 survey assessed reading and 
writing. Reeding prof ieiency as an objective was determined by con»^» bew» of its 
relevance and importance, measurement procedures were designed to assess reeding, and 
group performance data is available on time objectives. As ratal by Messiek ( 1 985): 

In cum, as currently conceived and implemented, NAEP incorporates three 
key elements of a responsible standard-setting proem -- namely, the choice of 
educational objectives, the description of current group performance ranps end 
trends, and the identification of educational contexts differentially related to 
performance. In edition, NAEP's reliance on comparative data and implicit 
comparative standards highlights the need for quality standards If issues of minimal 
requirements and excellence are to be resolved in American education. Educational 
progress, in an of itself , is a weak and insufficient standard (P. 11), 

NAEP reading proficiency data providsd an opportunity to explore the feasibility of 

conducting analyses of the relationship between minimum competency testing programs and 

student reeding proficiency from a rational perspective, 

Theoretical Perspective 

School systems are viewed as consisting of M nested layers" in which actions at the 

higher layers can help determine conditions in lower layers. The characteristics and 

conditions at the mate, district, community, and, school , levels interact to influence school 
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level functioning which ultimately affects teaching and learning In the classroom. Moreover, 
It Is generally the case that schooling achievement outcomes differ for various race/ethnic 
and socioeconomic groups. These differences are often attributed to family background or 
class differences, in studies that use a school production function mods) , statistical control of 
these factors are used to isolate properties of a school organization that facilitate students' 
learning and achievement. In this study , MOT programs were viewed as a potential "school 
effects" variable. 

Within this framework , numerous other variables which potentially affect the relation 
between MOT programs and reading outcomes must be considered at both the school and 
student level. For example, at the school level, factors which might be Included consist of 
regional variations in implementing reforms, grade- level differences, level of 
implementation, year of implementation, type of program, remedial programs, monetary 
resources for instruction, socioeconomic and racial composition of the school, and school 
retention and dropout rates. At the student level, race/ethnic group, age, family background, 
and time spent reading and on homework are potential factors Influencing studmt reading 
proficiency. 

Although, it was possible to control for several of these independent variables, it was 
difficult to construct a precise definition of the nature of a MOT program at either the state 
or school level. NAEP data were not collected to obtain detailed information on this particular 
issue. Thus, in addition to exploring potential relationships between MOT programs and 
student reading proficiency, a major question of interest was also in the feasibilty and 
utility of NAEP data to investigate and extreet such relationships. 
Purpose 

The studies reported here were exploratory in the use of a "school effects" framework 
and in the use of NAEP data The purposes were. 1 ) to investigate the relationship between 
school- level MOT programs and student reading proficiency within White, Black, and 
Hispanic groups, end 2) to investigate the relationship between discretion allowed local 
districts in Implementing MOT programs and student reading proficiency. The NAEP date 
provfoid a common measure of reading proficiency across a nationally representative ample 
of students at fourth, eighth, and eleventh grades, respectively. 



Sample 

The data for the present stud/ were taken from the 1983-84 assessment of the 
National Assessment of Educational Progress (NAEP), Each NAEP assessment has involved a 
random cross- sectional survey of in-school 9 ,13, and 17-year olds. In the 1953-84 
assessment, in eddition to sampling by &g$ t students of the corresponding modal grades 4, 8, 
and 1 1 were also sampled Each age/grade cohort included approximately 30,000 students. 
The NAEP sample was based on a highly stratified, three-stage sampling design in which, 
first, counties; second schools; and third, students were sampled In selecting schools, those 
in large cities with high concentrations of low sociooeconomic status students and those in 
extremely rural areas were samp led at twice the rate of other schools, 

In addition to the assessment of student-level data, NAEP collected school-level 
information concerning staffing patterns, curriculum, and student services from school 
administrators, The five-page questionnaire was completed by the principal or his/her 
representative, The survey response rat© for schools were 8 1 1 for grade 4/age 9 cohort, 
751 for grade 8/age 1 3 cohort, am! 751 for grade 1 1 /age 1 7 cohort. 
Subsample 

Schools included in all analyses were a non- random subsample of the original NAEP 
sample. Schools were included only if the principals or other personnel a) responded to the 
NAEP school questionnaire bjprovided responses to the MOT questions that met certain 
consistency criteria, and c) indicated that the MOT program had been implemented prior to 
1 980, The school response rates for the MOT item were 491 for grade 4 ( 521 for grade 8, 
and 601 for grade 1 1. The percentage of the total NAEP student sample included in the 
analyses included 39,81 at Grade 4, 41,81 at Grade 8, and 55,21 at Grade 1 1 . 
Procedures 

In the 1984 assessment, NAEP utilized a Balanced Incomplete Block (BIB) spiralling 
procedure, in which the total assessment battery was divided into blocks of approximately 1 5 
minutes each, Each student was administered a booklet containing three blocks as well as a 
six- minute block of general background questions. The balanced incomplete block part of 
themethod assigned blocks to booklets such that each block appeared in the same number of 
booklets and each pair of blocks appeared in at least one booklet. In the 1 984 assessment, 
57 different booklets for each age level were used The spiralling portion of the method then 
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cycles the booklets for administration so that to two students In any assessment session in a 
school received the same booklet. At each age group, each block Is administered to 
approximately 2000 stuebnts ami each pair of blocks to a smaller number depending upon 
the particular BIB design (NAEP, 1985), 
Reading EEflflclflQBE 

The measure of reeding proficiency employed In this stucty used item response theory 
( IRT) techrolog/ to estimate reeding proficiency levels, IRT defines a students 1 probability 
of answering an item correctly as a mathematical function of an under lying proficiency or 
skill The main objective of the IRT analysis is to provide a common scale on which 
performance can be compered across groups ami subgroups whether tested at the seme time 
or a number of years apart ( NAEP , 1 985), The sale also allows NAEP to make comparisons 
across age levels. The scale ranges from 0 to 500 with a standard deviation of 50. 

As the goal of NAEP is to estimate group means rather than Individual proficiency, each 
respondent may answer only a few of the total number of assessment items, Indicators of 
proficiency are computed as random draws from the expected distribution of proficiency of 
each respondent given the observed data, in this instance responses to NAEP reading sxercies 
and background variables. The distribution of such dram, one token for each respondent and 
weighted in Inverse proportion to the respondent's probability of appearing in the sample, 
estimates the distribution of proficiency in the population as a whole or in a given 
subpopulatim However, thu resulting values do not r e presen t precise estimates of 
proficiency for individual respondents. Five draws ere provided for each student who were 
administered at least cm block with reading item* 

Study 1 - Tha relationship tmtemm attflflLilfflffl] MOT end altftefll rmiiM 
proficiency 
Method 

!n stud/ 1 , analysis of cover fence within a multiple regression framework was 
conducted for each race/ethnic group (White, Black, and Hispanic) with each of the three 
grade cohorts assessed (grades 4, 8 .end 1 1 ), There were nine parallel rej^wsten equations, 
each incorporating the same predictor variables. Individuals' student reading proficiency 
was used as the dependent measure. Covariates at the student- level were age, sex, family 
background (a composite of parental education and possessions In the home) and students' 
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academic behaviors (a composite of amount of homework and number of pages read). 
Cevariates included at the school- level were region of the country, SES, end racial 
composition. The effect of minimum competency testing program was included as a 
school-level variable identified from principals' responses to the NAIP school questionnaire 
and was dummy eedtd to reflect the presence or absence of this program. School- level 
remedial program, and instructional dollars per pupil were included as two potential 
school-level explanatory variables, 
Results 

At grade 4, after controlling for student -level variables (sex, student's age, students' 
and parents' academic values) and school- level variables region, SES, racial composition, 
remedial program, and instructional dollars per pupil) there were no significant effects 
attributed to the school- level MCT variable for any of the race/ethnic groups. 

At grade 8, after controlling for the same student level variables, region, school -level 
SES, end racial composition there was a positive effect attributed to the school-level MCT 
variable for White and Black students, but not Hispanic students, This effect represented 
aboout an 8 (.29 s,d) point advantage for Whites and a 10 (.38 s.d.) point advantage for 
Blacks in mean reading proficiency as compared to their respective counterparts in schools 
indicating they had no MCT programs. The effect size (©lass, 1977) calculated as the 
difference between treatment and comparison adjusted means divided by the standard 
deviation of the companfian group is shown in parentheses. Effect sizes of .3 or greater are 
considered important (Cohen, 1977). The inclusion of a sehool-levf! remedial program and 
instructional dollars explained part but not all of the MCT effect. Inclusion of these variables 
reduced the effect for White students by about 291 end for Blade students by about 3 1 %. 

At grade 1 1 , after controlling for the same variables, there were significant positive 
effects for all race/ethnic groups. This effect represented a 2 (.06 s.d) point advantage in 
reading proficiency for White students attending schools with MCT programs, a 7 (.26 ad.) 
point advantags for Blacks, and a 6 (. 1 9 s.d.) point advantage for Hispanies as compared to 
their respective counterparts in schools without MCT programs. Inclusion of a school- level 
remedial program and instructional dollars per pupil statistically explained the effect for 
White students, explained a negligble portion of the effect for Hispanic students, end caused 
the effect for Black students to become larger. 



viii 

stint* 2 - The ralflUflnahln betwe en tvnn off state hct uaaam itateiri 

reading ornf tclgmrv 
Hathod 

In stud/ 2, school-level MOT responses were Identified by stole of origin for the 
purposes of classifying the type of state MOT mandate that existed at the school when NAEP 
was administered The system categorized school r esponses at yedes 4 end 8 according to: 
1 ) whether they were in a Mate controlled MCT program 2) a state MOT prwam that allowed 
local option(s) or 3 ) there was no state or 1ml mandate for MCT, but a state assessment 
program. At grade 1 1 , school responses were categorized according to 1 ) whether they were 
1n a state program that included MCT as a graduation requirement, 2) a state that al lowed the 
MCT as a graduation requirement as a 1ml option art 3) a state with no MCT graduation 
requirements. School responses (Yes-MOT ys No-MCT) were combined with the three 
categories of state programs which resulted in 6 potential celts in the design* For 
example, in call 1 were schools that had implemented a 1ml MOT program and were in states 
where there was a state-controlled MCT program* In cell 2 were schools that had 
implemented a local MCT program end were in states where there were local options available 
regarding MCT programs* In cell 3 were schools that had implemented a 1ml MCT prtqrom 
and were in states where the* £ was a state assessment program, in cell 4 were schools which 
reported no 1ml MCT program but were in states where there was a state-controlled 
mandated program* The number of re spon ses in this cstegpry were minimal and were 
excluded from the analyses, In cell 5 were schools with no 1ml MCT prqrams art in state 
which allowed 1ml options* Schools In rail 6 served as the reference category. These were 
schools that had rat implemented a local MCT projrem nor was there one required by the 
state* 

In this stud/, analysis of covariancc within a multiple regression framework waa 
eoH&jctad for each grade cohort. Individual student reading proficiency was used as the 
dspenctent measure. The covariates included at the individual student level were family 
background, and students' race. Covariates at the district level included the Orshansky 
percentile and at the school level , the percentage of students on free lunch. 
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Results 

At crede 4, after controlling for students' age, sex, parental academic values, race 
and district and school- 1**^ 1 5F C , the average rmdir^ proficiency of students in schools in 
which there was a state controlled MOT program was tower than students in schools in the 
reference group; i.e. , where there was? no MOT program at the school or state level 

At grade 8 , after adjusting for the same set of variables, the average reeding proficiency 
of students in schools in state- controlled MOT program was 9 (.31) points higher than 
students in schools in the reference group. In schools in which there was a local MOT 
program and where the state program included options, the difference in mean reading 
proficiency was 8 (,28 s,dj points. 

At grade 1 1 , after adjusting for the seme set of variables, there was a 7 (.22 s,d) 
point advantage in the average reeding proficiency of students in schools where MOT is a 
mandatory graduation requirement compared to students in schools in the reference {roup. 
There was a 6 (. 18 s,d) point difference in reeding proficiency between students in schools 
where MOT as a graduation requirement was a local option compared to students in the 
reference group. 
Discuss ten 

A mgjor purpose of study 1 was to investigate the relationship between school-level 
MOT programs and student reading proficiency within raw/ethnic groups. Students included 
in the subsample were a random sample, however schools included were based on 
self- report. Descriptive data examined for school and student- level variables by school 
questionnaire response suggest that the subsample of schools did not differ substantially 
from schools which did rot respond. However, ore must be cautious in generalizing results 
obtained to schools in the entire nation. 

Moreover, both studies were exploratory in wing data from NAEP to investigate and 
extract potential relationships between MOT projrama end student reeding outcomes. 
Students' reeding proficiency were estimates derived from IRT techniques Because these 
values are conditioned on certain variables; e,g., parental education, rare, region, age, and 
not otters, ag, schooHevel variables, the effects of nonronditioned variables are 
underestimated by about 15 to 201 when such variables are included in a regression 
analysis. Although the substance of any conclusions delved from this study would be 



essentially unchanged If all biases were removed 8 wry effect due to a nonconditioned variable 
such as, school-level MCT program, is likely to be extremely conserv ative when using NAEP 
data Conversely ( effects of conditioned variables; e.g, parents* education, raw, ere likely to 
be overestimated 

In study 1 , the effect of a school- level MCT program on student reading proficiency 
differed by grade and race/ethnic group. At fourth crade* after controlling for sex, age, 
region of the country, school context and SES and family background, and student academic 
behaviors, no significant effect due to MCT was isolated At eighth grade, a positive effect was 
found for White and Black students, but rot Hispanies, A portion of the effect for White and 
Black students was explained by the Inclusion of a specific school- level remedial program, 
and Instructional dollars per pupil. At eleventh grade, positive effects were found for ail 
race/ethnic groups. The effect for White students was statistically explained by school 
remedial program and per pupil instructional dbllars. These same variables, however, acted 
as suppressor variables in the regression equation for Blade studmts and explained little of 
the effect found for Hispanic students* This pattern suggests that certain variables may 
operate differently in different contexts. 

The failure to find en effect at fourth yette for any of the rece/ethnic groups might 
suggest that there ma/ be little or no advantage in implementing MCT programs at this gradi 
level In general , in elementary schools, there is an emphasis on instruction in basic skills, 
particularly reading, and perhaps the addition of a MCT program is superfluous, However, 
one possible explanation for no effect might be due to the unreliability of self-report data 
obtained at the fourth grade level, Additionally, schools includ» in this sample had the 
lowest response rate to the MCT questionnaire item. Since only one time point is being 
examined, the direction of causality between the variables cannot be established. Schools 
with MCT programs had students with lower reeding proficiency, and perhaps this situation 
resulted in schools Implementing a local MCT program, Additionally, there are many other 
important variables; e.g., academic engaged time, and content covered which are critical in 
explaining students' reeding proficiency end which were not included in these analyses. The 
results, however, suggest that additional studies should be conducted of the effects of 
school- level MCT programs on students' reading proficiency at the elementary level. 

The positive effects found at eighth gra* suggest that those schools with MCT may be 
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considered effective in facilitating students' reading proficiency. This effect may be due to 
the implementation of a MOT program. Alternatively , there may be other factors associated 
with the program but not measured, or other unmeasured school characteristics which 
contribute to this effect. Information was not available on school retention or dropout rates. 
An alternative explanation may be that schools who have institutional fad a MOT program 
have higher dropout and/or retention rates. The failure to find a MOT effect on the reeding 
proficiency of Hispanic eighth graders suggests that the variables included in the analysis 
may be insufficient in explaining proficiency of Hispanic students at this grade level. There 
are other variables which might influence proficiency for these students; for example, 
language dominance, language spoken in the home and in peer groups, and years of residaiee 
in the U.S. These results suggest the need to investigate effects of school- level MOT for 
Hispanic students at this grade level. 

At eleventh grade, the MOT effect on White students' reading proficiency could be 
explained by a school level remedial program and instructional dollars per pupil, For 
Blacks and Hispanics, this was not the case, Remedial program end instructional dollars 
were suppressor variables in the regression equation for Blacks, Other research suggests 
that remedial programs for MOT may be less effective in facilitating reading achievement of 
Blade students than for White students, However, remedial program In this study was a 
school-level rather than a student- level variable, and therefore cannot adequately address 
this issue. The results are merely suggestive that remedial programs in schools with MOT 
may have different effects on different groups. Instructional dollars per pupil was the 
amount spent on instructional materials only, and may not relate to total per pupil school 
expenditure 

The positive effects isolated at eleventh grade may be due to the inclusion of a 
school- level MOT program. Alternatively, the effect may be due to other unmeasured 
characteristics of these particular schools. As suggested in the dlaeuslson of eiphth grade 
results, information on school dropout or retention rates was not available in NAEP. There 
is the increased possibility that those students who are doing poorly or have failed a MOT are 
no longer in the school at eleventh grade. In general , Black and Hispanic dropout rates are 
higher than those of White students. The dropout rates, end possibly average reeding 
proficiency might be higher in schools that have institutionalized MOT programs. 
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Study 2 

The objective of study 2 was to Investigate the relationship between the type of state 
program end reading proficiency. The receding of the initial school responses provided 
additional information on policies of state mandated programs regarding discretion allowed 
to local school distrf eta Schools were categorized according to whether state programs were 
completely controlled at the state level, there were some options available, or there was a 
state testing program only. Different effects were found by craft level and type of state 
mandate. At fourth grade, students in schools with state controlled MOT programs had 
slightly lower reading proficiency compered to students in schools in which there was no 
state or local MOT program. At eighth end eleventh grafts, there were positive effects due 
to a state- control led MOT program ; however, there were similar positive effects due to state 
MOT programs in which local options were allowed. This pattern suajests that at the upper 
grade levels, the two types of state programs have about the same effect on students' reeding 
proficiency. 

The discrepancy between fourth grade, and eighth and eleventh grade results may be 
due to differentia! influences of the various types of state programs on the school reading 
curriculum. One study indicated that In districts that use test- management strategies 
similar to MOT, the mode of teaching changes from performenee^orlented activities; ag,, 
reading books, discussing ideas, to test-oriented activities; e,g*, fill-in the blank 
worksheets, memorization of facts and drill or rote skills. Time strategies may be 
detrimental to elementary students* reading pr^iciawiifwtheyMsttll learning how to 
reed at this level Moreover, them elementary schools characterised as successful in 
facilitating basic skills exercise a considerable amount of discretion in solving, problems 
locally, e.g. , school^ wide staff development , and frequent and careful monitoring of students' 
progress. It may be the me that externally- mandated programs operate against this type of 
local discretion and negatf vely influences st udent reading proficiency outcomes, 

In eighth grade, students attending schools with local MOT programs and in which the 
state maintained controlled had higher reading proficiency compared to students in the 
reference group. Similarly , students in schools with a local MOT program and where 
there were local options had higher reading proficiency compared to students in the reference 
group. At the eighth grade level , the implementation of a school or state- level program that 
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emphasizes the attainment of basic skills may provide the opportunity to learn skills that 
students failed to acquire at the elementary level, in general, during the middle and junior 
high grades, schools are generally departmentalized and organized around various subject 
matter areas, it may be that the MOT programs at this level orients the schools and teachers 
to continuing basic skill development for particular groups of students. The finding that 
regardless of the type of state MOT program (state controlled vs local option), students at 
this grade had higher reading proficiency outcomes compared to the reference group might 
support this notion. 

in eleventh grade, students attending schools with a local MOT program and in which 
successful completion of the program was a state graduation requirement had higher 
reading proficiency compared to students in the reference group. Students in schools with a 
local MOT program end in which MOT a s state graduation requirement was optional also 
had higher reading proficiency compared to students in the reference group. A»1n, one 
possible explanation may be that schools and teachers ere including additional opportunities 
for continued skill development for students who previously failed to acquire such skills. At 
this particular grade level; however, the sample may be more selective than at the other 
grades due to dropout rates. 

!n gener ul , the results of study 2 suggest that it may be possible to isolate differential 
effects of state policies regarding minimum competency tasting programs on student reading 
proficiency at various grade levels. 
Summary and ftmclnriflna 

In conclusion, the results of the two exploratory studies conducted using NAIP data 
suggest it is possible to isolate a relationship between MOT programs end student reading 
proficiency as an outcome. Based on analyses presented in these studies, this relationship 
appears to differ depending on the grade level, race/ethnic group, and type of state program 
assessed. At the elementary level, school -level MCT programs did not appear to be related to 
students* reeding proficiency as an outcome. In addition, at this grade level, there was a 
small negativa relationship between reading proficiency and MCT programs administered and 
controlled at the state level. 

Ataighlh grade, positive effects of school- level MOT programs were isolated for WK.'te 
and Black students, but not Hispanic stutfents. The effects of state-controlled MCT programs 
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and state MOT programs which allowed local options appeared to be equally effective at 
producing higher student reading proficiency. Similarly, at eleventh grade, positive effects 
of school- level MOT were isolated for each race/ethnic group. Students in schools in states 
requiring MOT as a graduation requirement, or in states allowing this as an option were found 
to have higher reading proficiency compared to students in states where there was no state or 
local MOT required at high sehoo* 

The remaining discussion will present some of the conceptual and methodological 
limitations of the studies presented here. First, the definition of a MOT program was 
initial V limited to a schools' response on the NAEP questionnaire regarding whether or not 
they had a MOT program, and the year the program was implemented There was little 
information available on the exact nature of the local school program, it would seem 
important to know the purpose, additional personnel or curriculum used, whether ths MOT 
was commercially prepared or locally prepared, the content, and performance standards, and 
the proportion of studmts in the schools failing to meet requirements, retention and dropout 
rates. Future NAEP questionnaires might include 2-3 additional Items which would 
characterize in more detail the nature of a particular school" level MOT program. 

Second, school- level MOT is a proxy variable for other Important school 
characteristics. Although additional descriptive information on the nature of the program 
might be obtained in a large-Male survey, the "process" variables and interactions within 
schools which might accompany the implementation of a MOT and influence students' reeding 
proficiency; e,g,, teachers' expectations, opportunity to learn, might not be adequately 
derived from such measures. Additional qualitative studies of schools implementing MOT 
programs would be useful in this area 

Third, this stud/ was conducted from data collected at one point in time, end as such 
presents the status of students' reading proficiency in schools at that time period, in order 
to assess change In students' reading proficiency due to MOT, one must nacsBas'ily have date 
over two or more time periods, and preferably for cohorts of students, ideally, one would 
want to look at the same schools, or same students ever two or more time periods tc 
investigate a potential school effect due to the implementation of a MOT program. This type of 
information is not within the design or scope of NAEP , and other longitudinal date bases might 
be explored. 
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Fourth, school organiatiens are viewed as consisting of "nested layers" In any 
explanation of student outcomes, it is necessary to consider teachers and classroom practices 
as well as school- level variables. Classroom or teacher variables were not directly 
examined in this study. Moreover, individual student- level data concerning participation in 
MOT and remedial programs would be preferable to using school - level aggregates of such 
variables, in conducting multi- level analyses of school data, we know that many of the 
assumptions band on aggregate data do not reflect reality- e.p,, pupils actually receive 
differential exposure to school resources and facilities. This may be especially true of MOT 
programs in general, and remedial programs, more specifically, individual participation 
might include measures of additional engaged time or content covered in such programs. It is 
only with such data that the question of effectiveness of MOT for various groups of students 
can be adequately assessed 

Fifth, additional variables potentially related to Hispanic reading proficiency in 
schools with MCT at eighth grade need to be explored. There variables might include such 
student- level character istics,- e.g., languge dominance, years of residence in the U.S., 
participation in remedial reading or other special programs for language minority students. 

Sixth, results from these studies should be replicated using other available national 
data bases. The unique psychometric techniques employed by NAEP in deriving individual 
reading proficiency estimates; i.e., conditioning on background variables, provide accurate 
population and subpopulation estimates of reading proficiency. However, when both 
conditioning end nonconditiened variables ere included in a regresion analysis, the effects 
due to noficonditioned variables are underestimated, in this core, by about 1 5 to 20%. Thus, 
results presented in these studies provids extremely conservative estimetes of the school- 
end state policy level MOT effect 

Finally, the categoriiation system used in study 2 might be refined to obtain a more 
precise operational definition of discretion available to local school districts. It may be that 
the number of available options is important or that different options; eg,, the schools' 
ability to select a test aligned with the curriculum more relevant to reading 

proficiency outcomes. Other classifications of state policies regarding MCT, for example 
whether or not a MCT program is used for local remediation funding or for grade-to -grade 
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promotion might also differentially influence students' reading proficiency. 

The diversity of local and state policies related to MOT make It extremely difficult to 
characterize precisely the nature and outcomes of this educational reform in the nation. 
Although NAEP data vwe not designed to specifically address the particular issues explored in 
these studies, the analyses end results demonstrate both the limits and potential of using 
NAEP data to inform national policy issues. Additional studies wing NAEP and other national 
data bases are needed to further explore art refine relations investipted in this study, 
Moreover , these studies will provide information necessary to determine whether educational 
reforms, such as MOT are actually improving students' reading proficiency and educational 
standards in the nation, rather than merely improving students* scores on minimum 
competency tests end also contributing to national *epout statistics, 
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SECTION I 
INTRODUCTION 

Overview of \\m Problem 

Since the mid-seventies, over thirty-five states have required local school districts to 
give minimum competency tests to students in elementary, junior high, or senior high school 
(Pipho, 1 983). In 1 984, forty states were actively involved in some aspect of minimum 
competency testing (MOT); nineteen states were using test performance as a basis for high 
school graduation, and five states were using tests as a basis for grade promotion (Anderson & 
Pipho, 1984). The legislation fa* MOT can be viewed within the larger context of the 
accountability movement in education which focuses concern upon the output of the 
educational system (Wise, 1 977). 

Wise ( 1 97?) suggests that there are two different problems which MOT is designed to 
solve. The first addresses the declining value of a high school diploma. Soma students who 
graduate from high school Jai* basic skills in reeding, writing, and mathematics, MOT is 
designed to ensure that high school graduates have the basic skills necessary to pass a state 
or local examination. The second problem hoped to be addressed by implementing MOT is to 
influence educational practice by improving the eoquisition and teaching of basic skills of 
students who fell to acquire these skills, and for those teachers who fail to teach the basic 
skills (Wise, 19??). 

The purpose of what MOT hopes to accomplish is clear. However, the underlying 
resumptions concerning how that aim is to be achieved do not consider the process of 
education. The focus of MOT is on the clarification of the goals of schooling; i.e. , basic skills 
of reading and math, and assessing performance of these goals with teste. Wise (197?) 
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states: Thus, educational policy Is designed to alter the practice of education without sn 
understanding of how education actually acurs" (P.22), The assumption underlying MOT 
appears to be that legislating goal attainment is sufficient for such attainment to occur 
(Wise, 1977). Districts, schools, and classrooms across the nation have been required to 
respond to this policy. 
What is Minimum Competency Testing 

Minimum comptency testing may be defined as a program to teat students in terms of, 
and only in terms of, whatever competencies state or local authorities have decided are the 
minimally acceptable outcomes of an education (Lazarus, 1981), As a result, minimum 
competency is what minimum competency tests assess. There are few commonly agreed upon 
definitions of the exact nature of minimum competency testing programs. States differ 
dramatically with respect to economic climate, educational expenditures, and policies. Among 
the many policy differences is what is labeled and characterized as a MOT prwsn. MOT 
programs vary substantively as well as procedurally* Substantive differences include the 
purpose, content, arid grade levels assessed. Procedural differences include the agency that 
sets the performance standards, the determination of passing scores, and the degree of 
discretion allowed local school districts. There are numerous published listings that 
summeriis state MCT across these various dimensions (Pipho, 1981, 1983; PI iskoi* Stern, 
1985) end reports which examine MOT programs in depth (Qoertz, 1985; Gtorlh & Perkins, 
1979; Pipho, 1979, 1980). A comprehensive review of a"' of the substantive and 
procedural dimensions will not be provided here, (See Jaeger & Tittle, 1 980 for a discussion 
of various dimensions, models and consequences of MOT)* However, a brief review of the 
content of MOT programs, and the discretion allowed to local school districts Is provided 
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Co ntent of MCT 

There we wide discrepancies 1n skills tested In minimum competency testing praams 
(Lazarus, 1981). The majority of siates assess reading and mathematics (Pipho, 1978, 
1 984). However, there is little consensus regarding the assessment of "life skills" (Lazarus, 
1 98 1 ). These skills might include those that would enable individuals to cope with practical 
problems in everyday life; e.g., health and nutrition or consumer rights. Alternatively, 
"life skills" might Include school-related tasks such as applying skills learned in math to 
calculate a sales tax or balance a checkbook. Chall ( 1 983) indicated that the commercial 
publishers of MCT prefer to include real life content, whereas the states are divided, half 
preferring basic skills and half preferring life skills. 

Even If we were to assume that the majority of state mandated programs focus on basic 
skills, the difficulty level of the content of MCT programs varies from state to state. One 
analysis found considerable differences in competency level (defined by mew passage 
difficulty) among 8 minimum competency tests in reading administered to 1 1th graders 
(Chall, 1 983). The various permutations and manipulations used to derive passing scores 
only exacerbates the problems of comparing outcomes which results from state MCT 
programs. The differences between programs in each state make it virtually impossible to 
assess the impact of MCT on other than a intra- state basis. 
Discretion Available to Lo cal Districts 

Moreover, tffferenees between state MCT programs involve differing requirements, 
options, regulations, end guidelines to local school districts in implementing such programs. 
These differences result in varying degrees of discretion available to local school districts. 
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As noted In the introduction, minimum competency testing was not Initiated by educators, but 
largely by those outside of the educational system, As a result, there was little concern for 
the active participation or ownership on the part of school personnel Beyer ( 1 985) note 
that as more authority shifts sway from the local school , we may be shaping a bureaucratic 
education mo*l that leaves teachers and principals more accountable, but less empowered 
Among the various state programs, there is variation in the degree of discretion available to 
local districts in implementing ere! administering MOT programs. For ©cample, in some 
state programs, the purpose, the performance levels, the tot, and the grade levels assessed, 
are determined at the state level; ag. New Jersey. In otter state programs, the actual test, 
the performance levels, grade levels assessed are left up to the local district For ©cample, 
in California, the state requires students to pass a test to graduate but allows the local 
districts to choose both the test and the passing score, 

The level of implementation of a state MOT program in a particular school may vary as 
a function of the amount of local discretion allowed Change can be successfully 
implemented at the individual school building level given the appropriate conditions, 
procedures, and support systems (Fullen, 1965; Ooodlad,1984). The school may also be 
viewed as a unit of decision making with its own incentive structures (Darling-Hammond & 
Wise, 1 985), Thus, in tightly-controlled state MOT programs, principals and teachers who 
are actually responsible for students 1 learning may have a lower degree erf ownership and 
less involvement in implementing state- mandated testing programs. In these particular 
schools, the effect of MOT programs on students" learning outcomes may differ from that 
found in schools functioning with more discretion; e.g., selecting the test, grade level, or 
whether or not to Implement the state program. Schools that are successful in facilitating 
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basic skills in reacting have been characterized by such school- level factors as, principals' 
instructional laadBrship, monitoring of student and teacher progress, high teacher 
expectations for students" achievement, curriculum articulation and organization, and 
school-wide staff development (MacKenzie, 1983; Purkeyi Smith, 1983; 1988;Yenezky 
StWinfield, 1979). 

Darling- Hammond I* Wise ( 1 985), in a discussion of state policies suggest that policies 
should be "reticent " in nature since they rely on technical end political implementation 
through many layers of bureaucracy. They state- "Where technologies are uncertain and 
means-end connections are tenuous, the use of an axe to perform heart surgery may kill the 
patient" (P.333). Thus, one important dimension of MOT programs to be considered in 
assessing student outcomes may be the degree of discretion allowed local districts and schools. 
Rationale 

Among the many unresolved issues surrounding MOT are, for example, the effect on the 
curriculum, teachers and students' expectations, and the achievement of low SES and 
minority students, Todate, Information regarding outcomes of MOT derive primarily from 
intra-atate assessment data. There is little or no information regarding the extent to which 
MOT programs nave led to improvement in student reading achievement and higher 
educational stsfidards in the nation. Investigations of these issues require an objective 
measure of achievement common to all states. Data collected in the 1963-84 National 
Assessment of Educational Progress (NAIP) may provide some insight on these issues. 

NAEP is an on-going oongressfonatiy- mandated project established to report on the 
educational achievement of American students. Since 1969, when NAEP was established, 
national assessments were conducted annually up until 1980, From 1980 to the present, 
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assessments have been conducted biennially, The 1983-84 survey assessed reeding and 
writing. Reading proficiency as an objective was determined by consensus because of its 
relevance and importance, measurement procedures were designed to assess reading, end 
group performance data Is available on those objectives. As rated by Messlck ( 1 985); 

In sum , as currently conceived and implemented, NAEP incorporates three key elements of 
a responsible standard-setting proem -- namely, the choice of educational objectives, the 
description of current group performance ranges and trenft, and the identification of 
educational contexts differentially related to performance. In addition, NAfP's reliance on 
comparative data and implicit comparative standards highlights the need for quality stanctards 
if issues of minimal requirements and excellence are to be resolved in American education. 
Educational progress, in end of itself , is a week and insufficient standard (P. 11). 

(See Messiek, 1 985 for a discussion of the role of NAEP results in clarifying and assessing 
educational stanftrds and engaging the public with these issues). Although NAEP drta were 
not specifieal \y collected to investigate the issue of MOT programs and student outcomes, the 
1983-84 NAEP provide an opportunity to explore potential relationships from a national 
perspective. 

Theoretical Perspective 

In this stud/ school systems ere viewed as consisting of "nested layers* in which actions at 
the higher layers can help determine conditions in lower layers (Purkey & Smith, 1983), 
The characteristics and conditions at the state, district, community, and school levels 
interact to influence school level functioning which ultimately affect? teaching and learning 
in the classroom. Moreover , it is generally the case that school achievement outcomes differ 
for various race/ethnic and socioeconomic groups These differences are often attributed to 
family background or socioeconomic status. In studies which use a school production function 
model, statistical control of thro factors are necessary to isolate properties of a school 
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organization that facilitate students* learning and achievement. 

In this study. MOT programs were viewed as a potential school effects variable. Within 
this framework, there are numerous other variables that potentially affect the relation 
between MOT programs and student reading outcomes. For example, school factors which 
might be considered important include regional variation, level and year of implementation, 
resource allocation, remedial programs, racial and socioeconomic composition, retention, 
and dropout rates. At the student- level, factors, such as race/ethnic group, family 
background, time spent reading and on homework, and grade level also potentially Influence 
reading achievement outcomes. Although it is possible to control for several of these 
independent variables in analyses, it is difficult to construct a precise definition of the 
nature of a MCT program at the school or state level from the available data In conducting 
secondary analyses, questions are posed and available data explored in a post hoc fashion, 
NAEP data ware not collected specifically for this purpose. In addition to exploring potential 
realationsriips between MCT programs end student reading outcomes, another objective was 
to explore the feasibility and utility of NAEP data to inform national policy issues. 
ElUJHBB 

The studies reported here were exploratory in the use of a "school effects" paradign , and 
the use of NAEP data to. 1) to Investigate the relationship between school-level MCT 
programs and White, Black, end Hispanic student reeding proficiency as assessed by NAEP, 
and 2) to investigate the relationship between discretion allowed local districts in 
implementing state MOT programs and student reeding proficiency. The NAEP data provide a 
common measure of reeding proficiency across a nationally representative sample of 
students at fourth, eighth, and eleventh grades, respectively. 
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SECTION H 

DESCRIPTION OF SAMPLE, PROCEDURES, AND VARIABLES 

Sample 

The data for the present stud/ ere from the 1983-84 assessment of the National 
Assessment of Educational Progress (NAEP). Each NAEP assessment has involved a random 
cross-sectional survey of in-school 9 ,13, and 17-year olds. In the 1983-84 assessment, 
in addition to sampling by age, grades 4, 8 and II were also sampled. Each age/grade cohort 
included approximately 30,000 students, The NAEP sample was based on a highly stratified, 
three-stage sampling design in which, first, counties,* second schools; and third, students 
were sampled. In selecting schools, those in large cities with high concentrations of low 
sooiooeeonomio status students and those in extremely rural arm were sampled at twice the 
rate of other schools. Less than SI of the students sampled were excluded because of 
limited-English proficiency or a severe handicap. 

In addition to the assessment of student- level data, NAEP collected school- level 
information concerning staffing patterns, curriculum, and student services from school 
administrators, The five-pflge questionnaire was completed by the principal or his/her 
representative. The overall survey response rates were 81 % for grade 4, 751 for grade 
8, and 751 for grade 1 1. 
Subsample 

Schools included in this study are a non-random subsample of the original NAEP sample. 
Sdiools we™* included only If the principals or other personnel: a) responded to the school 
questionnaire* and b) provided responses to the minimum competency questions that met 
certain eonsis^inoy CKteria. ' The school response rate for the item requesting information 
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on minimum competency testing were lower than overall survey response rates and ware 
491 for grade 4, 521 for grab 8, end 601 for grade 11. The unweighted and sflustad 
weighted frequencies of schools and students In the subsample ere shown In Table I, 2 
Procedures 

In the 1984 assessment, NAEP utilized a Balanced Incomplete Block (BIB) spiralling 
procedure, In which the total assessment battery was divided Into blocks of approximately 1 5 
minutes each. Each student was administered a booklet containing three blocks as well as a 
six- minute block of general background questions. The balanced Incomplete block part of the 
method assigned blocks to booklets such that each block appeared in the same number of 
booklets end each pair of blocks appeared in at least one booklet In the 1 984 assessment, 
5? different booklets for each age level were used. The spiralling portion of the method then 
cycles the booklets for administration so that no two students In any assessment session 1n a 
school received the same booklet At each age group, each block is administered to 
approximately 2000 students and each pair of blocks to a smaller number depending upon 
the particular BIB design (NAEP, 1985), 
Student Level Variables 

Readi no Proficiency The measure of reading proficiency employed In this study uses 
item response theory ( IRT) technology to estimate reading proficiency levels. IRT defines a 
students' probability of answering an item correctly as a mathematical function of an 
underlying proficiency or skill The main objective of the IRT analysis is to provide a 
common rale on which performance ran be compared across groups and subgroups whether 
tested at the same time or a number of years apart (NAEP, 1985), The scale also allows 
NAEP to make comparisons across age levels, 
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As the goal of NAEP is to estimate group means rather than individual proficiency, each 
respondent may answer only a few of the total number of assessment items. Indicators of 
proficiency are computed as random draws from the expected distribution of proficiency of 
each respondent given the observed data, in this Instance responses to NAEP reading exercles 
and background variables (see Mislevy, 1985, for the statistical foundations of this 
approach). The distribution of such draws, one taken for each responAnt and weighted in 
inverse proportion to the respondents probability of appearing in the sample, estimates the 
distribution of proficiency in the population as a whole or in a given subpopulation. 
However, the resulting values do not represent precise estimates of proficiency for 
individual respondents, Five draws are provided for each student who were administered at 
least one block with reeding items (approximately 851 of each cohort). 

The NAEP reading proficiency scale ranges from 0 to 500. Five levels of reading 
proficiency were identified on the proficiency scale: rudimentary* (150) basic (200), 
intermediate (250), adept (300) and advanced (350), Thro levels were defined by the 
kinds of reading tasks that most readers at each level would be able to cb and are based on the 
complexity of the passage, the familiarity with the subject matter , and the kinds of questions 
asked ( NAEP p 1 985), 3 A brief description of each level can be seen in Appendix A. 

Family Background. A composite variable was formed which included responses to 
items on parental education, reading materials in the home, and the extent of family reeding. 
Parental education was assessed on a four-point scale: did not finish high school, graduated 
from high school ; went on to another school after high school and graduated from college. In 
cases where answers for both parents were available, the highest level of the two parents 
was used. Cases where answers for both parents had teen omitted were maintained separately 
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since a substantial number of the younger respondents did not respond to either question, 
items measuring reading in the home consisted of en affirmative response regarding the 
presence of a dictionary, an encyclopedia, more than 25 books and whether or not the family 
receives newspaper and magazines regularly. Items measuring extent of family reading 
consisted of the frequency of family members reading the newspapers, magazines, and 
books. The response scale included five categories ranging from "almost every day" to 
"never or hardly ever". The composite variable was calculated as the sum of two or more 
affirmative responses to any of the items included in the set of family background items. The 
composite ranged from a low value of 1 to a high value of 9. Alpha reliability coefficients 
for this composite were .47 for grade 4, .50 for grade 8, and .59 for grade 1 1 . 

Students' Academ ic Behaviors. A composite variable was formed which consisted of 
Items requesting the number of pages read for school and the amount of homework. The 
item on number of pages read- About how many pages a day, do you have to read in school and 
for homework? The response categories were "5 or fewer", "6-10", "1 l-1i" "16-20" or 
"more than 20". The homework item read: "How much time did you spend on homework 
yesterday? The response categories "no homework was assigned" and "had homework but 
didn't do it" were combined. The remaining three categories were maintained separately; 
"less than I hour", " 1 -2" hours", and "more than 2 hours". The composite variable ranged 
in value from a low value of 1 to a high value of 8. Alpha reliability coefficients for this 
composite were .10 for grade 4, .25 for grade 8, and .46 for grade 1 1 . 
School Laval Va riables 

School-level variables included from the NAEP school characteristics questionnaire were 
categorized as policy, demographic, school composition, and socioeconomic variables. 
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School Polity School- policy variables consisted of whether or not a MOT procrem 
existed at the school, and if so whither a specific remedial program existed for students 
failing the test in reading. 

Minimum C ompetency Testing Program. There were four questions on the NAEP 
"school characteristics and policy questionnaire" related to MOT. Two of the items which 
requested information on MOT in reading were used in this study. One item read: H ln which 
of the following subjects ere students required to pass a minimum competency test?" 
Respondents were required to answer either # Yes M or "no" to items identifying several 
subject areas, one of which was reading. The second item read: ! 1n what year was each of the 
following minimum competency tests first administered?" Affirmative responses to the 
MOT item on reeding were then coded according to whether or not the program had been 
implements prior to 1 980, This group formed the majority of the responses. 4 

School- level responses to the MOT questionnaire item were identified by state of origin 
for the purposes of delineating type of state MOT program, 5 These responses were 
categorized according to whether or not: 1 ) there was a state- mandated MOT program which 
was, to a large extent controlled at the state level .ag,, the state sets the performance 
standards, develops and administers tests, deoifcs the grade levels; 2) there was a state 
mandate; however , there were options available at the local level ; e.g. , grade level assessed, 
teat used, or whether or not to implement a MOT program- and 3) there yj-as no state or 
local mandate for MOT, but a state assessment program. 

School responses In Orade 1 1 were classified similarly but with the additional 
information of whether or not there was a proficiency, exit, or minimum competency test 
required for graduation at any of grades 9 through 1 2, When the school level questionnaire 
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responses (MOT yes/no) are combined with the three types of state programs, there are six 
possible combinations of school program types or 6 calls in the design. 

For grades 4 and 8, schools in cell 1 are those that responded "yes" to the MOT item on 
the NAEP survey, and originated from states where MCT programs are controlled at the state 
level. Schools in cell 2 are those schools that responded "yes" and originated from states 
where there were options availeble regarding the state mandate. If one option is whether or 
not to implement a program , these schools opted to do so. Schools in cell 3 are those that 
responded "yes" and originated from states where there is a state testing program only. These 
schools are those that may be Involved in a local competency testing initative. Schools in cell 
4 are those schools that responded "no" and originated from states where MOT programs ere 
controlled by the state. These schools represented fewer than 5» of the total number of 
responses at each grade and were excluded from all subsequent analyses. Schools in cell 5 
are those that responded"no" and originated from states where local options are available. 
If the option was whether or not to implement a MCT program, these particular schools did 
not. Schools in cell 6 represent these that responded "no" and originated from states where 
there is a state testing program only. 

For grade 1 1 , schools in cell 1 are those that responded "yes" and originated from a state 
where a proficiency test was mandatory as a graduation requirement at any of grades 9 
through 1 2. Schools in cell 2 are those responding "yes" and originated from a state where a 
proficiency test as a graduation requirement was a local option. Explanation of the other 
categories for grade 1 1 follow the same logic as described for grades 4 and 8. Additional 
details of the procedures used in categorizing responses and the listing of state- mandated 
programs by state developed for ceding purposes are ahown in Appendix B. (NOTE: All 
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state are shown on the listing but all state were not included in the NAEP sample, schools 
were sampled in approximately 38 of th# state,) The adjusted weighted frequencies end 
percentage of schools end students by type of state program are shown in Table 2, 

Demographic Variables . Demographic variables included the four geographical 
regions of the country included in NAEP: northeast, southeast, central, and west The NAEP 
categories of "Size end Type of Community" were examined for descriptive purposes, A brief 
explanation of these oategories is presented in Appendix (X 

School Composition and Socioeconomic Variables School composition consisted 
of the percentage of White, Black v and Hispanic students in a school Sccioeeonomic variables 
examined included the Orshensky percentile, the percentage of students in a school receiving 
free lunch, and instructional dollars per pupil The Orshansky index measures the 
percentage of students under the poverty guideline as a percentage of total school age children 
in the district Instructional dollars is a measure of spending on Instructional materials, 
This measure; however, may not relate to the total level of spending in a school 
Instructional dollars per pupil was originally measured categorically with each category 
providing a range of expenditures. These categories were recoded to reflect the midpoint of 
the interval: 1^$ 14,99, 2^$ 19,99, 3=$29,99, 4=$39.99, 5^$49,99, 6^$S9.99, 
7^$69.99and 8^$79,99. 

Characteristics of Schools with MOT Programs 

Because analyses were based on a subsample of the total NAEP sample, school-level and 
student- level descriptive data are provided for each grade level by school type 
(MOT-yw/MCT-no) and for each race/ethnic group included in the study. 

erode 4, A n examination of the percentage of schools by response type indicated that 
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schools with MOT programs were distribute equally across each of the four regions, Schools 
without MOT programs ware more likely to originate from the Central region and least likely 
to originate from the Southeast. Schools which did not respond to the MOT item ( no^response 
group) were distributed equally across each of the four regions, Schools In all three (roups 
were from diverse types of communities as represented by the NAEP categories of "Size and 
Type of Community 0 , However, there was a larger percentage of "disadventagid urban* 
schools included in the sample of schools with MOT programs as compared to the percentage 
included In the sample of schools without MOT programs, On the average, schools with MCT 
programs had larger school enrollments, and a lower percentage of white students compared 
to schools without MCT end schools in the no- response groups. Schools with MCT programs 
also tend to have specific remedial programs for students falling reeding compered to schools 
without MCT programs. Demographic data and weighted school means by response groups 
for Grade 4 are shown in Table 3. 

Grode 8, An examination of the percentage of schools by response type indicated that 
schools with MCT were likely to be distributed equally across each of the four regions. 
Schools without MCT programs were more likely to originate from the Central region and 
least likely to originate from the Southeast Schools in the no response group were 
distributed across all four regions. Schools in all three groups were from diverse types of 
communities as represented by the NAEP categories of "Size and Type of Community". 
However, similar to fourth grade, there was a larger percentage of "disadvantaged urban" 
schools included in the sample of schools with MCT programs as compared to the percentage 
included in the sample of schools without MCT programs, On the average, schools with MOT 
programs had a lower percentage of white students compared to the schools without MCT 
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programs and schools in the no- response groups. Schools with MOT programs tend also to 
have specific remedial programs for students failing ruling compared to schools without 
MCT programs. Demographic data and weighted school means by response groups for Grade 
8 are shown In Table 4, 

Srate 1 1 ^ An examination of tha percentage of schools by response type indicated that 
schools with MCT were more likely to originate from the southeast and western regions end 
least likely to originate from the Central region, Schools without MOT were more likely to 
originate from the Central region and least likely to originate from the Southeast Schools in 
the no -response group were from all four regions; however more likely to originate from 
the Central region, Schools in all three groups were from diverse types of communities as 
represented by the NAEP categories of "Size and Type of Community"* On the average, schools 
with MOT programs schools had larger school enrollments and a lower percentage of white 
students. Schools with MCT programs tend to also have specific remedial programs for 
students failing reeding compared to schools without MCT programs. Demographic data and 
weighted school means by response groups for Grade 1 1 ere shown in Table 5. 
ChPHSterlstlES of Schools with MCT Programs Bv Rac a/Ethnlc Groups 

The magnitude of the differences in the school composition and school - level SE3 
variables among raw/ethnic groups suggest that there are substantial differences in the 
nature of the schools attended by Whites, Blacks, and Hispaniea regardless of school type. 

At all three grade levels and in both schools with and without MCT programs, on the 
average, Black and Hispanic students were more likely to attend schools in which there was a 
higher proportion of low-SES students, and a lower percentage of White students, floertz 
( 1 985) in a description of the total NAEP sample found a similar pattern. She indicated that 
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a White student was more likely to attend a school where the student body is 801 or more 
White whereas a Black or Hispanic student was most likely to attend a school where more 
than 501 of the students are minority. In this subsample, there was more variation in SES 
and school composition variables among race/ethnic groups than between types of schools, or 
within each race/ethnic group. As might ba expected, at all three grade levels and for all 
race/ethnic groups, the average instructional dollars per pupil average was higher in schools 
with MOT programs compered to schools without such programs. 
Charapterlstics of Stude nts bv Rage/Ethnic flroup 

At grade four, the average parental education was similar across types of schools and 
race/ethnic groups. On the average, fourth grade students reported parents* education as 
category 3 "going on to school after high school" The average parental education reported by 
Hispanic students was lower thai that reported by White end Black students. However, there 
is the possibility that the similarity across race/ethnic groups may be due to less accuracy 
in fourth- graders , self report. There was a higher proportion of missing deta for fourth 
graders 1 reporting of parental education (361) compared to the otter grate ( 1 1 1 at grade 
8, 51 at grade 1 1 ) and it Is possible that when younger students do not know their parents* 
educational level, they bias their answers upward. However, there was a greater disparity 
among raw/ethnic groups when possessions in the home were included to form the family 
background composite. On the average, this composite was higher for White students then for 
Blacks or Hlspanics. 

At both eighth and eleventh grate, parents' education was slightly higher for students 
attending schools with MOT programs as compered to schools without such programs. White 
students reported higher averages of this variable compared to Black students, who reported 
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slightly higher averages than Hispanics. Similar to the pattern in fourth grate, there was a 
greater disparity among race/ethnic groups in the composite variable "family background" 
as compared to reported parents 1 educational levels, The average for White students was 
higher than that of Blacks and Hispanics. 

In both eighth and eleventh grades, students attending schools with MOT reported higher 
averages of student academic behaviors (homework + pages read) as compered to students in 
schools without MOT programs. This pattern was consistent across race/ethnic groups. The 
averages for school- level and student-level variables in the no-response sample fell within 
the ranges of averages reported for the two types of schools. School-and student-level 
descriptive date by raw/ethnic group for grades 4, 8, and 1 1 are shown in Tables 6, 7, and 
8, respectively. Comparable data for the schools that did not respond are shown in Appendix 
D. 

Unadjusted Reading profi ciency bv raco/othnic group by tchool type 

The achievement gap between minority and non minority students has besen documented 
( NAEP 1 985), NAEP's report on reeding proficiency among the nation's 9 P 1 3 , and 1 7 -year 
olds Indicated that despite pins by minority students in the last 10 years, Black and 
Hispanic students continue to read at a significantly lower level than White (NAEP, 1 985), 
Figure 1 depicts the average reeding proficiency by grade by race/ethnic group for the total 
NAEP sample. 

In the subsample of students included In this study, the gap in reading proficiency 
between minority and non minority students exists at all three grade groups and regardless of 
school type. However, this study and the discussion of reading proficiency focuses on 
comparisons of reading proficiency within race/ethnic group by school type. Unadjusted 



42 

ERIC 



18 



reading proficiency means and standard deviations by grade by rare/ethnic group by school 
type are shown in Table 9, 

firede 4. In general, at fourth grade, the average unadjusted reeding proficiency of 
stuttsnts attending schools with MOT programs Is lower then or similar to that of students 
attending schools without MOT programs, For White and Hispanic students, this difference 
represents S (.16 s.d) and 4 (,13 s.d,) points in reading proficiency, respectively. For 
Black students, there was no difference In the average read1,rg proficiency of students 
attending either type of school. 

On the average, White and Hispanic students in schools without MOT programs and In 
which there was a specific remedial program for students falling the reading tests had 
slightly higher reading proficiency compared to their respective counterparts In schools 
with MOT and a specific remedial program. However, Black students In schools with MOT and 
no specific remedial program had slightly higher reading proficiency compared to Black 
students In schools with MOT and a specific remedial program, (NOTE: Remedlel program Is 
a school-level variable and information was not obtained at the student level Thus, student 
proficiency is an aggregate of all students within a school and not just students who receive 
remedial instruction). Unadjusted reading proficiency by school type by remedial program 
for each grade level is shown In Appendix E . 

An examination of NAEP proficiency levels by school type indicated that for all reee/ethnie 
groups, In schools with MOT programs, there were substantial proportions of students 
reading at or below the basic and rudimentary levels as compared to the proportion of 
students reading at or below these levels in schools without MOT programs, and in schools 
that did not respond (Of course, we do not know how much of the proportion of students at 
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the lower reading levels may be due to retention or other factors). The distributions of 
fourth grade students performing at or below each NAEP proficiency levels by school type 
are dtpicted graphically for each reee/ethnle group in Figure 2,3, and 4, 6 Sample sizes 
and percentages for these groups are presented In Appendix F, 

Brade ft. At eighth grade, the average reading proficiency of students attending schools 
with MOT was higher than that of students attending schools without MOT programs. For 
White students, this difference represented 7 points (.26 s,d,); for Blade students 1 2 points 
( ,44 s.d.), and for Hispanic students, 8 ( ,26 s,d,) points in reeding proficiency. 

On the average, White and Black students in schools with MOT and a specific remedial 
program had higher reading proficiency compared to their respective counterparts in schools 
with MOT and no specific remedial program. For Hispanic students, average reeding 
proficiency was similar In schools with MOT regardless of whether or rat there was a 
remedial program. 

An examination of NAEP reading proficiency levels by school type at eighth grade indicated 
that there was a lower proportion of students reeding at or below the basic and intermediate 
levels, and a higher proportion at or below the adept and advanced level in schools with MCT 
programs as compared to the proportion of students reading at these levels in schools without 
MCT programs. This pattern was evident for White and Blacks, but not Hispanics, For 
Hispanic students, there was a slightly higher proportion of students at or below the 
advanced level but also a higher proportion of students at or below the basic level In schools 
with MCT programs as compared to similar proportions in schools without MCT programs. 
The distributions of eighth graders performing at or below each NAEP proficiency levels by 
school type are shown for each race/ethnic group in Figures 5, 6, and 7, 
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flrorie 1 l a At eleventh grade, ths average reading proficiency was higher In schools 
with MOT programs for all race/ethnic groups, For White students, there was a 7 (.23 3.1) 
point difference, for black students a 5 (.18 s,d.) point difference, and for Hispenics a 7 
(.22 s.d.) point difference. 

On the average, White and Hispanic students in schools with MOT and in which there was a 
remedial program had higher reading proficiency compared to their respective counterparts 
in schools with MOT and no remedial programs. Black students in schools with MOT but with 
no remedial programs had slightly higher reading proficiency compared to Black students in 
schools with MOT and a remedial program. 

An examination of NAEP reading proficiency level by school types indicated that for 
White students, there was a higher proportion of students reading at or below the adept and 
advanced levels and fewer students at or below the basic and intermediate levels in schools 
with MOT programs as compared to the proportion of students at these levels in schools 
without MOT programs. For Black students, in schools with MOT programs, a similar pattern 
was evident for students reading at or below the adept level For Hispanic students, the 
distributions of students by reading proficiency levels were similar regardless of school 
type. The distributions of eleventh graders performing at or below each NAEP proficiency 
levels by school type are shown for each race/ethnic group In Figures 8, 9, end 1 0, 
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SECTION III 

STUDY 1 - THE RELATIONSHIP BETWEEEN SCHOOL-LEVEL MOT 
PROGRAMS AND STUDENTS* READ I NO PROFICIENCY 

Purpose 

There wars two major questions of interest in the study. First, what is the 
relationship between school- level MOT programs and students 1 reading proficiency within 
racial groups? In the analysis related to this question, the reading proficiencies of students 
in each race/ethnic group were examined after adjusting for students' age and sex, region, 
school- level 3E5, family background, and students' academic behaviors, school- level MOT, 
remedial program, and instructional dollars per pupil Second, what is the relationship 
between student reading proficiency and type of state MOT program as defined by the 
discretion allowed to local districts? In these analyses, students* reeding proficiencies for 
each grade cohort was examined after adjusting for the type of state program , student age, 
sex, school and district- level SES, family background, and race. 
Design 

In conducting multilevel analysis of schools, there is no consensus among axial 
scientists concerning the appropriate unit, Pedhaiur (1982) note that the issue is not 
merely one of choice of an appropriate unit but also conceptualizing and developing analytical 
approaches that wi«l make full use of the different types of information contained In the 
different levels. In general, results obtained from regression analyses using 
individual- level data will differ from those obtained in analysts using aggregate as the 
unit of analysis (Bursteinli Miller, 1981), 

In stud/ 1 , school level data were merged with individual student common background 
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items and reading proficiency data Individual reeding proficiency was used as a ctependent 
measure Thi question of interest was the relationship between school- level MOT program 
and ^oog proficiency within raw/ethnic groups. From a statistical framework, one might 
ask ( w^at proportion of total variant In reading proficiency; e.g, , among black students, can 
be ^counted for by school-level MCT programs after controlling for SES and background 
factors? And, are the effects of MCT the same for ell reoe/ethnie group? The question posed 
in the second atutf/ might be phrased similarly; i.e. , what is the proportion of total variance 
in student proficiency accounted for by various types of state programs, controlling for 
school- level SES, and other factors? 

ideally, to answer either of these questions we would w nt to look within schools at 
individual student level achievement to examine changes in the distribution of reading 
proficiency for various race/ethnic and SES groups over a period of time. In this study, we 
are limited to the available data for one time period to investigate relationships between 
schools and student reading proficiency. 

We know that many of the assumptions made when using multilevel data in an analysis 
do not reflect reality. For example, school level aggregates are assumed to affect all students 
equally but we know that pupils receive differential exposure to school resources and 
facilities. This may be especially true with respect to MCT, since most programs ere 
targeted toward students who, by some criterion fail to acquire basic skills in reading. 
Moreover, resources and facilities per se do rat influence reading proficiency, but how 
schools use such resources and facilities (Madaus, Afrasian & Kei1aghan f 1980), 
Other limitations to this approach ere reflected in the operational definitions of the 
independent and dependent variables. In study 1 , the independent variable, MOT is based on 



23 



principals* self-report, so that we have no Vnowledge of specific characteristics of the 
program or whether or not a MOT program exists at the school, Similarly, the depen*nt 
measure, individual students 1 reading proficiency Is an estimate of proficiency, and as such, 
includes a certain degree of error, 7 These studies are therefore viewed as exploratory in two 
areas: 1 ) investigating the relationship between MOT as a potential school effect and 2) using 
the MAEP data to inform policy issues, 
Subsample Size 

The adjusted weighted sample sizes for grades 4, 8, and 11 were 10,367 ,10,829, end 
13,513. These numbers represent 39.81 of the total KAEP 4th grade cohort, 41,81 of the 
Grade 8 cohort, and 55,21 of the Grade 1 1 cohort 8 The number in each racial/ethnic group 
was 7,491 Whites, 1 ,733 Blacks and 1 ,143 Hispanlcs in grade 4; 7,574 Whites, 1 ,906 
Blacks end 1,349 Hispanlcs in grade 8, and 9,203 Whites 2,112 Blacks and 2,198 
Hispanlcs in grade 1 1, 
Data Analysis 

The purpose of study 1 was to investigate the relationship between school- level 
mlnlnu.n competency toting program and student reading proficiency. An analysis of 
oovariance within a regression framework was conducted for each rare/ethnic group (White, 
Black, and Hispanic) with each of the three grade cohorts. This model assumes that the 
within-group regression coefficients are homogeneous for schools with and without MOT 
programs, and that one may test differences between groups after adjusting for the effects of 
other attributes. There were nine parallel regression equations each incorporating the 
same predictor veriebles, Individuals 1 student reading proficiency was used as the dependent 
measure* Oovariates at the student level were age, sex, family background, and students* 
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academic behaviors, Govariates include at the school level were region of the country, 
percentage of students on free lunch, Qrshsansky percentile 9 , and racial composition. The 
school-level MOT variable was dummy coded, 1*yes and O s no, School-level remedial 
program for reading (dummy coded, l-yes, and O^no) and Instructional dollars per pupil 
were included as two potential school-level explanatory variables. Variables were entered in 
the following order: 1 ) student age and sex, 2) region of the country, 3) school level SES and 
composition 4) family background 5) student academic behaviors, 6) school-level MOT 
program and 7) school level remedial program, and 8) Instructional dollars per pupil 

All possible interactions were not tested; however, the interactions between the MOT 
variable and the variables; student age, school-level SES, region, MCT, percentage of 
students on free lunch, percentage of white stuctents, family background, and students* 
academic behaviors were tested as a block, entered last, and were found to be nonsignificant. 
AH regression analyses were conducted on students in the grade samples (rather than age 
samples) which included approximately 781 of each age/grade cohort. Listwise deletion of 
missing cases was used in all analyses, * § 
Results 

Table 1 0 presents the unstandardized regression weights and standard errors for the 
MCT dummy- coded variable obteinad from the regressions of individual student reeding 
proficiency on school- and studentHevel variables for each race/ethnic group in each 
grade, 1 1 The first column presents the effect of MCT after aflusting for sex, age, region of 
the country, school- level SES, family background, and students 8 academic behaviors, The 
second column presents the effect after adjusting for all of the student and schooHleval 
variables in addition to the explanatory variables, per pupil instructional dollars, and 



school-level remedial reading pro-am. 

At grade 4 , after controlling for student, school- level and explanatory variables, there 
were no significant effects attributed to the MOT dummy variable for any of the race/ethnic 
groups. At {rede 8, after adjusting for student and scftool-leve. variables, there was a 
positive effect for both White and Blade 8th graders. This effect represented about an 8 ( .29 
S-d) point advantage for White and a 10 (,38 s.d) point advantage for Blacks in average 
reading proficiency as compared to their respective counterparts in schools without MOT 
programs. The effect size (Olass, 1 977) as calculated by the difference between treatment 
and comparison adjusted means divided by the standard deviation of the comparison group Is 
shown in parentheses. Effect sizes of ,3 are greater are considwed important (Cohen, 
1 977). The inclusion of a school-level remedial proc^em end Instructional dollars per pupil 
explained part but not all of the MOT effect Inclusion of these variables reduced the effect 
for White students by about 298 ami for Blade students by about 311, Mo signficant effect 
was isolated for Hispanic students. 

At grade 11, after controlling for student end school- level variables, there were 
positive effects for all rare/ethnic groups. Thfe^effect represented a 2 COS s,d.) point 
advantage in average reading proficiency for White students attending schools with MOT 
programs , a 7 ( ,26 s.d ) point advantage for Blacks , and a 6 ( . 1 9 s,d ) point advantage for 
Hispanics as compared to their respective counterparts in schools without MOT programs. 
Inclusion of 8 school- level remedial program and instructional dollars explained the effect 
for White students, accounted for a negligible portion of the effect for Hispanic students, and 
caused the effect for Black students to become larger. 1 2 Zero order correlation coefficients 
for the MOT dummy-coded variable and reading proficiency are shown in Appendix H 
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SECTION IV 

STUDY 2- STATE MOT PROGRAMS AND STUDENT READING PROFICIENCY 
Design 

The purpose of stud/ 2 was to investigate the relationship between school^ level MOT 
progams and type of state mandate. Type of state program was itentified by state of origin and 
responses were classified as: 1 5 state mandate for MOT controlled at the state level , 2) state 
mandate with local option(s) or 3) state assessment program wily, Listwise deletion of 
missing cases was used in all analyses. There were 1 2,36 1 students included in the analyses 
of grade 4, 16,719 In grade 8, and 1 8,326 in grade 11, The racial composition of each of 
these samples ran be found in Appendix 0, 
Data Analysis 

Analysis of eovarience within a regression framework was conducted for each grade/age 
cohort Individual student proficiency was used as the dependent measure. The covarlates 
included at the individual student level were family background, and students* race (dummy 
coded 1 -White, 2^B1§ok f and Hispanic as other), Covarlates at the school level included 
school and district- level SES. The type of state program was dummy coded for each cell 
included in the design with students in schools having no state or local level MOT program 
coded as other. Type of state program was entered first, art control variables entered 
hierarchally in blocks. 
Results 

The purpose of study 2 was to investigate the relationship between MOT and type of state 
program. The unadjusted average reading proficiency of students in the subsample attending 
schools located within the various types of state programs are presented in Table 1 1 . The 
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unstandardized regression coefficients are presented in Table 1 2, 13 These effects have been 
adjusted for students' age and sex , district, and school SE5, family background, and race, 

As shown In Table 12, at grade 4, after controlling for student and school-level 
variables, the average reading proficiency of students In schools In which there was a state 
controlled MOT program was slightly lower than students In schools in the reference 
group Jhis affect represented a 4 (A2 s.d) point difference in mean reading proficiency, 
Students in schools with local MOT programs and in which there was a state testing program 
also had slightly lower reading proficiency compared to students In schools In the reference 
group. This effect represented a 3 (. 1 1 ad) point difference in mean reading proficiency. 

At grab 8, after adjusting for student and school-level variables, the average reading 
proficiency of students where there was a state control led MOT program differed by 9 (.31 
s.d) points as compared to students in schools in the reference group, In schools in which 
there was a MOT program and in which the state allowed local options, there was an 8 (,28 
ad) point difference in mean reading proficiency as compared to the reference group. 

Smaller positive effects were found for students in schools in the other categories. 

At grade 1 1 f after adjusting for student and school- level variables, there was a 7 ( ,22 
s.cL) point difference In the average reading proficiency of students in schools where MOT was 
a mandatory graduation requirement compared to students in schools in the reference group, 
There was a 6 ( , 1 8 s.d. ) point difference in reading proficiency between students in schools 
where MOT as a graduation requirement was a local option compared to students in schools in 
the reference group. 
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SECTION V 
DISCUSSION 

A major purpose of study 1 was to Investigate the relationship between school-level 
MOT programs and student reading proficiency within race/ethnic groups. Students included 
In the subsample were a random ample, however schools Included were based on self-report 
Descriptive date presented for school and student-level variables by school response s*jggBst 
that the schools included in the study did not differ substantially from schools which 
provided no response to the NAEP questionnaire item concerning MOT, One must be cautious, 
however a in generalizing results obtained to schools in the entire nation. 

Moreover, both studies were exploratory in using data from NAEP to investigate and 
extract potential relationships between MOT programs and student reeding outcomes. The 
measures of students 1 reading proficiency were estimates derived from IRT techniques. 
These values were conditioned on certain variables; &g,, parenj^l education, raoe, region, 
age, and not others, &g, suhooHevel variables. When NAEP reeding proficiency estimates 
are used as outcome measures and both conditioning and nonconditioned variables are 
included in a regression analysis, the effects of nonconditioned variables are underestimated. 
In this study, this undsr^timate is in the range of 15 to 20% <5ee Note * 7). Although the 
substance of any conclusions derived from this study would be essentially unchanged if all 
biases were removed, any effect due to a nonconditioned variable such as, school-level MCT 
program, is likely to be extremely conservative when using NAEP data. Conversely, effects 
of conditioning variables; ag, parents 1 education, race, eta are likely to be overestimated 

in study 1 , the effect of a schooHevel MCT program on student reading proficiency 
differed by grade and raoe/ethnic group, At fourth grade, after controlling for sex,agB, 
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region of the country* school context and SES end family backer ound, and student academic 
behaviors, no significant effect due to MOT was isolated. At eighth grade, a positive effect was 
found for White end Black students, but rot Hispanics. A portion of the effect isolated for 
White and Black students was explained by the Inclusion of a specific school-level remedial 
program, and instructional dollars per pupil At eleventh grade, positive effects were found 
for all race/ethnic groups. The effect for White students was statistically explained by 
school remedial program and per pupil instructional dollars. Thro same variables, 
however, acted as suppressor variables in the regression equation for Blade students and 
explained little of the effect found for Hispanic students. This pattern suggests that certain 
variables ma/ operate differently In different contexts. 

The failure to find on effect at fourth grade for any of the race/ethnic groups might 
suggest that there may be little or no advantage In implementing MOT programs at this grade 
level In general, in elementary schools, there is a general emphasis on instruction in teic 
skills, particularly reading, and perhaps the addition of a MOT program is to some extent 
redundant. However, one possible explanation for no effect might be due to the unreliability 
of self-report data obtained at the fourth grade level Additionally, schools Included in the 
fourth grade sample had the lowest response rate to the MOT questionnaire item. Since only 
one time point Is being examined, the direction of causality between the variables cannot be 
established. Schools with MOT hod students with lower reeding proficiency, and per hops this 
situation resulted in schools implementing a local MOT program. Additionally, there are 
many other important variables* &g., classroom practices, academic engaged time, and 
content covered which are critical in explaining students* reading proficiency and which were 
not included in these analyses (Winfield, 1987eX The results, however, suggest that 
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additional studies should be conducted of the effects of MOT programs on students' reading 
proficiency at the elementary level. 

One possible explanation for the positive effects found at eighth end eleventh grade is 
that by restricting the variance in background factors to within groups, the probability 
increased of demonstrating an effect due to school variables, (This should also hold for fourth 
grade as well). Other studies, using a similar method, have found that among schools attended 
by students of the seme SIS background or race, it was possible to identify some that were 
consistently "effsctlve or Ineffective" (Fradiriksen, Boft, Beranek, and Newman, 1975). 

Those schools with MOT may be considered effective in facilitating eighth grade 
students* reading proficiency. This effect may be due to the implementation of a MOT 
program. Alternatively, there may be other factors associated with the program but not 
measured, e.g., monitoring of student progress, or other unmeasured school characteristics 
which contribute to this effect. The distributions of White and Black students' NAEP 
proficiency levels in schools with MOT programs were shifted upward for all students, and 
not just for those students at the lower reading proficiency levels. This might suggest that 
the effect may not be due solely to MOT, but to other school programs and characteristics in 
those schools with MOT programs. However, information on school retention or dropout rate 
was not available. It may be that schools that have institutionalized a MOT program have 
higher levels of retention and/or dropout among lower-performing students. 

The failure to find a MOT effect on the reading proficiency of Hispanic eighth graders 
suggests that the variables included in the analysis may be insufficient in explaining reading 
proficiency of Hispanic students at this grade level. There are other variables which might 
influence proficiency for these students- for example, language dominance, language spoken 
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In the home end in peer groups, and years of residence in the US. Alternatively, the school 
variables included may operate differently in different contexts, Hanushek ( 1970) found 
differences In teachers and classrooms related to the achievement of White students, but not 
Mexican students. These results suggest the nied to Investigate effects of school-level MOT 
for Hispanic students at this grade level 

At eleventh grade, the MOT effect on White students' reading proficiency could be 
statistically explained by the inclusion of school level remedial program, and instructional 
Altars per pupil For Blades and Htspanlos, this was rot the cm Remedial program and 
instructional dollars were suppressor variables in the regression equation for Blades, Based 
on unadjusted average proficiency. Blade students in schools with no specific remedial 
reeding progr am performed as well on the NAEP reading proficiency scale as Black students 
in schools with remedial programs. Other research suffpsts that remedial programs for 
MOT may to less effective in facilitating reading achievement of Blade students thai for 
White students (Sorrow, 1984), However, remedial program In this stud/ was a 
school- level rather than a student- level variable, end therefore cannot adequately address 
this issue. The results ere merely suggestive that remedial programs in schools with MOT 
may have different effects on different croups, 

The positive effects isolated at eleventh grade for each ran/ethnic group may be due 
to the inclusion of a school-level MOT program. Alternatively, the effect may be due to other 
unmeasured characteristics of these particular schools, As suggested with respect to 8th 
grade results, there is the increased possibility that those students who are doing poorly or 
have failed a MOT are no longer in the school by 1 1th grade, In general, Blade and Hispanic 
dropout rates are higher than those of White students ( NCES, 1 985), If there is a greater 
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degree of exclusion and selection in schools with MOT programs, then one mighl expect the 
average reading proficiency to he higher in those schools. The results obtained for the 
eleventh grade me/ pertain to a more select population compared to fourth end ei^ith grades, 
This may be especially true of minority roups. Burton and Jones ( 1 9«2) reached a similar 
conclusion regarding NAEP data for 1 7 year olds, in a comparison of achievement trends of 
Black and White youth, they suggested that it would not be possible to assess whether the 
relative improvement observed in the Blade population at ages 9 awl 1 3 persisted at age 1 7 
because of the differential dropout rate by race end sex* They indicated that an 
out-of-school, 17-year old sample would be needed to assess the trend There is also 
evidence that 258 of students who initially failed a state MOT program withdrew from 
school prior to graduation (Sorrow, 1 984), 

The objective of study 2 was to explore the relationship between the type of state 
program and reading outcomes. The receding of the initial school responses provided 
additional Information on policies of state mandated programs regarding discretion allowed 
to local school districts. Schools were cate^prlzed according to whether state programs were 
completely controlled at the state level, there were optima available, or there was a state 
Ming program only. Different effects were isolated by grade level end type of state mandate, 

in 4th grade, after adjusting for achooHand studentHevei factors, students attending 
schools with MOT program and in which the state had control had slightly lower reading 
proficiency compared to students in schools with neither of these characteristics* Students in 
schools with local MOT programs end in which there was a state testing program only also 
had lower reading proficiency compared to students in schools in the reference group, 
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In eighth grade, students attending schools with local MOT programs end In which the 
state maintained controlled had higher reading proficiency compared to students in the 
reference group. Similarly, students in schools with a local MOT program with state-allowed 
options had hither reading proficiency compared to students In the reference group, Smaller 
positive differences In reading proficiency were found for students in schools in which there 
was a local MOT program and a state testing program only, and for students in schools in 
which there was no local MOT program end the state program allowed options, 

In eleventh grade, students attending schools with a local MCT program and in which 
successful completion of the program was a jraduatlon requirement tad higher reeding 
proficiency compared to students in schools in which neither of these characteristics were 
present Students in schools with a local MOT program aid in which MOT 83 a graduation 
requirement was optional also had higher reading proficiency. 

In general, the results of stud/ 2 suggest that it may be possible to isolate differential 
effects of state policies regarding minimum competency testing on student reading 
proficiency- At fourth grade, students in schools with state control led MOT programs had 
lower reading proficiency compared to students In schools 1n which there was no state or 
local MOT program, At eighth and eleventh grades, there were positive effects due to a 
state- control led MCT program ; however , there were equally positive effects due to state MOT 
programs In which local options were allowed, This pattern suggests that at the upper grade 
levels, the two types of state programs may have similar effects on reading proficiency. 

The discrepancy between fourth grade, and eighth and eleventh grade results may be due 
to influences of the various types of state programs on the school reading curriculum. One 
study indicated that in districts that use test- management strategies similar to MCT, the 
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mode of teaching changes from performance-oriented activities; e.g., reading books* 
discussing ideas, to test-oriented activities; e.g, , fill-in the blank worksheets, memorization 
of facts and drill or rote skills (Darling- Hammond and Wise, 1985), These strategies may 
be detrimental to elementary students' reading proficiency since they are still learning how 
to read at this level Moreover, those schools characterized as successful in facilitating basic 
skills exercise a considerable amount of discretion in solving problems locally (Purkey & 
Smith, 1 983), It may be the case that externally- mandated programs operate against this 
type of local discretion at the elementary level and negatively Impact student reading 
outcomes. Alternatively, it is often at fourth grade that students* reading achievement 
results begins to decline, The small negative effect found due to type of state program may be 
due to a general decline in student achievement at this grade level, 

In eighth grade, the students attending schools with local MOT programs and in which the 
state maintained controlled had higher reeding proficiency compared to students in schools 
with neither of thro characteristics, Similarly, students in schools with a school MOT 
program and where there were local options also had higher reading proficiency, At this 
grade level, the implementation of a school or state- level program that emphasises the 
attainment of basic skills may provide the opportunity to learn skills that students failed to 
acquire at the elementary level, Traditionally, during the middle and junior high grades, 
schools are generally organized around various subject-matter areas. It may be that the 
MOT programs at this level orients the schools end teachers to continuing basic skill 
development for particular groups of students, Alternatively, results may be due to higher 
retention rates in schools implementing MOT programs. 

In eleventh grade, students attending schools with a local MOT program and in which 



59 

ERIC 



35 



successful completion of the program was a state graduation requirement had higher reading 
proficiency compared to students in the reference group. Students in schools with a local 
MOT program and in which MOT as j state graduation requirement was optional also had 
higher reeding proficiency. Again, one possible explanation may be that additional 
opportunities are being provided for basic skill development for students who previously 
failed to acquire such skills. At this particular grade level; however, the sample may be 
more selective than at the other grades due to dropout rotes. 
Summary and conclusions 

In conclusion, the results of the exploratory studies conducted using NAEP data suggest 
that it is possible to isolate relationships between MCT programs and student reading 
proficiency outcomes, The relationship appears to differ depending on the gretfe level, 
race/ethnic group, and type of state program assessed. At the elementary level , school- level 
MCT programs appeared to have no significant effects on reeding proficiency for my of the 
race/ethnic groups studied. Those MOT programs administered and controlled at the state 
level appeared to have a slight negative impact on students 1 reeding proficiency at this level. 

At eighth grade, positive effects of school- level MCT programs were isolated for Whits 
and Black students, but not Hispanic students. The effects of state-controlled MCT proyams 
and state MCT programs which allowed Iocs! options were related to higher student reading 
proficiency. Similarly, at eleventh grade, positive effects of school-level MCT were isolated 
for each race/ethnic group. Students in schools in states requiring a MCT for graduation, or 
in states where this was an option were found to have higher reeding proficiency compared to 
students in schools where there was no state or local MCT required at high school 

The remaining discussion will present conceptual and methodological limitations of the 
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studies. First, the definition of a MOT program wm iitftHfy limited to a schools* response 
on the NAEP questionnaire regarding whether or not tteyW innplemented a MOT program 
and the year in which the program was initiated. T^ww little detailed information 
available on the exact nature of the local school program. It wwjld seem Important to know 
the purpose, special personnel or curriculum used- y^lter %h& MOT was commercially 
prepared or locally prepared, the content end perform standards, the proportion of 
students in the schools failing to meet requirwianWi rotation and dropout rate, Future 
NAEP school questionnaires might consider Including 2^3itlor»l Items to obtain additional 
detailed information on the nature of the school of stflte teal MOT prosraffK It is generally 
the case that most of the available large-scale data behave been less than optimal for 
cofKlueting certain types of policy analyses C Plisko 8* Oit^burg k Ohaikend, 1 985), 

Second, school-level HCT is a proxy veTitfrt for other Important school 
characteristic Although descriptive information wi tte natures of the program might be 
obtained in a large- scale survey, the process variables and t nteraccions within spools 
which might accompany the implementation of a Mel and influence students 1 reeding 
proficiency; e.g., teachers' expectations, opportunity to learri ^ might not be adequately 
derived from such measures. Additional qyalitativ© stwtfesof schools implementing 1ml and 
state MOT programs would be useful in this area. 

Third, this stud/ was conducted from data collects one point in time, and as such 
presents the status of students* reeding proficiency in %ctrtet that time period In order 
to assess chaixp in students' reading proficiency due to MOT, one must have Ma over two or 
mere time periods, and preferably for cohorts of stgtf&riteThis *ype of information is not 
within the design or scope of NAEP , end other lor^itudir^i to bases might be explored 
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Fourth, school organiatfons are viewed as cornsisting of nested layers, In any 
explanation of student outcomes, it Is necessary to constckBsr teachers and classroom practices 
as well as school -level variables. To some extent, the gffwcts of thro variables are subsumed 
In an overall school effect, however, teacher or classrMDomHevsl variables deserve direct 
examination, These were no! within the scope of tahis study. Moreover s individual 
stucfentHtvel data on participation in MCT and remedial programs would be prefer able to 
using school-level eggregatesof such variables. In concrJuctlng multilevel analyses of school 
data, we know that many of ^assumptions of using aggregated data do rot reflet reality, 
e.g,, pupils actually receive differential exposure to §ci=fK>ol resources and facilities. This 
may be especially true of "at-risk" students (Winfie1d M in press) who are likely to be in 
remedial programs as a result of MCT, individual stetudent participation might Include 
measures of additional academic engaged time or content teovered In such programs. It Is only 
with individual student- Java! participation data that the question of effectiveness of MOT for 
various groups of students can be adequately assessed. 

Fifth, additional variables potentially related to Hispanic reading proficiency In schools 
with MCT programs at eighth grade need to be txplorid ~ These variables might include such 
student- level characteristics; e.g., language dominancft^ years of residence in the U, 5,, 
participation in remedial reeding or other special pragraavris for languge minority stuctents. 

Sixth, results from these studies should be replie^sted using other available national 
data bases. The psychometric technique employed by N^EP provide accurate population and 
subpopulation estimates of reeding proficiency. Hcwewer, In regression analyses using 
reading proficiency estimates as outcome measures, ormmd Including both conditioning and 
nonconditlnoed variables, the regression effects due to noncondltloned variable are 
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underestimated. In this particular study, this underestimate mounts to 1 5 to 20%. Thus, 
results presented in the studios presented here provide extremely conservative estimate of 
the school-and state policy level MOT ef fect. 

Finally, the categorization system used In study 2 might be refined to obtain a more 
precise operational definition of discretion available to local school districts, it may be that 
the number of available options is important or that different options; e,g,, the schools' 
ability to select a test aligned with the curriculum, might be more relevant to reading 
proficiency outcomes, Other classifications of state policies regarding MOT, for example 
whether or not a MOT program is used for remediation fundi ng or for grate to grade 
promotion might also differentially influence students' reading prof ioiency. 

The diversity of local and state policies related to MOT make it extremely difficult to 
characterize precisely the nature and outcomes of this educational reform in the nation. 
Although NAEP data were rat designed to specifically address the particular issues explored in 
these studies, the analyses and results demonstrate both the limits and potential of using 
NAEP data to inform national policy issues. Additional studies using NAEP and other national 
data bases are needed to further explore relations investigated In this study, Moreover, these 
studies will provide Information necessary to determine whether educational reforms, swh 
as MOT are actually improving students' reading proficiency and educational standards in 
the nation, rather than merely Improving students' seoris or passing rates on minimum 
competency tests and also contributing to national dropout statistics. 
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MOTES 

1 Approximately 1 -28 of the responses at each grade/age gra#p were inconsistent and 
were not Included in subsequent analyses, Inconsistent responses were respondents who 
answered M no H to the first MOT question used end subsequently completed the second MOT 
Item on grade implemented or the third Hem on yw Implemented. 

2 Beeeuae the NAEP sample design employ stratifications wid clustering (students 
within schools, schools within PSUs), the resulting sample fws different statistical 
characteristics from three of a simple random sample. To account proximately for the 
effects of the sample design, a design effect of 2 was used. This hes *hs effect of dividing the 
sample size In half and using the adjusted sample slid In the commutation of errors. This 
method was used in lieu of the ETS jacknife technique employed In estimating sampling 
variability of statistics included In NAEP reports, Because NAEP produces a representative 
national sample, each student or school has an associated sampling weight to account for the 
differential probability of selection and oflustments for non-response and 
post-stratification. To insure adequate representation, certain subw^oups were sampled at a 
filler rate than the rest of the population. Thus, In all analyses » a student's or school's 
weight was adjusted so that the sum of the reseated weights equalled the number of cases 
included in each analysis. This rescaled weight was divided by two < the design effect). All 
analyses reported tare were conducted using adjusted weights. (See NAEP User's Guide, June 
1 986 for procedures to be used when analysing NAEP data). 

3 In the scale anchoring process, NAEP salaried sets of Items that were good 
discriminators between proficiency levels* The criterion used to iderrtlfy such items was that 
students at any given level would have at least an 801 probability of success with those 
reading tastes while students at the next lower level would have less Than a 501 probability 
of success ( NAEP , 1985). 

* At grade 4 , 7% of the school responses fell in the category of 1 inplementing MOT after 
1981 p at grade 8, 4.81 and at grade 1 1,4.21. 



40 



D A 1 0S random sample of Wheals that did not respond to the MOT item were also 
identified by state. Results of this informal analysis yields no consistent pattern of 
particular states that did not respond. 

6 In figures 2 through 10 depleting NAEP proficiency Bevels, the percentage of 
students depicted were those who* racing proficiency estimates fell within the following 
score intervals: 1-150= (at or Mr jRudimentary, 151-2^00 (at or below) Basic, 
201-250 (at or below) Intermediate, 25 1 -30 (at or below) ASept, and 301-500 (at or 
below) Advanced, 

7 The dependent variables used In the regression analyses repported here are ^plausible 
values" for reading proficiency froffl the 1983/84 NAIP rr-«ding assessment. The 
instruction and properties of these variables ere described in def-tall in the NAEP Technical 
Report (NAEP, 1986), One Property is particularly pertinent tea the present paper. Item 
response theory was employed in tw reading assessment, leading to the NAEP Reeding rale 
upon which the results here and else^tta have been reported, lemo few items are presented 
to each respondent, however, to allow (lie estimation of proficienoies for each individual in 
the sample, "Plausible values* are intermediate steps in the evaluation of integrals that yield 
consistent estimates of selected menjlnsof the national population; specifically, these are 
gsncter, ethnicity, parents* education* si and type of community, rage, region of the country, 
and grade. These may be referred to $ the M conditioning varieties, * Analyses involving 
other background variables are subject to regression effects in regression analyses, 
regression coefficients for beckgr ognq variables otter thai the conditioning variable tend to 
be underestimated by 1 5-201, and coefficients for conditioning wariebles may be inflated. 
These effects are present to an extent that depends on the nature and strength of the 
relationship among conditioning variabi, norKBndltioned variables, and reading proficiency 
(Misley, 1 986, personal cofnrnunicatfofi), Details and numerical Illustrations are included 
in the Technical Report (NAEP, 1386), 

8 These percentages reflect the number of students after: 1 ) excluding schools which 
implemented a MOT program after (951,2) students whose reoiaWethnic classification was 
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not White - , Black or Hispanic, and 3) students who wire ndeAninlstered a reeding block , and 
4) stud^nt^s who wri not in the grade sample. Since the a^^npl« also encompass moat of 
each oohor — t (approximately 751), the overlap between thapreds and age samples are great 
(approxirrwataly 541), In other words, over half of iach cohort \m In both the age and grade 
sample ' This means that the analysis do not differ substantially between limiting the sample 
on the totts of grade or age. However s it is incorrect to basothe analysis on the entire cohort 
since the ^cohorts include both students who are overagodde., olcfer than the modal age of 
students fm their grade) and under graded (I.e., in a lower grade than the modal yade of 
students ox-f the same age). Both of these groups have lower riding proficiency than students 
in the mo^el grade and age group, thus they downwardly bias the proficiency level for the 
cohort. F: his downward bias is even more exaggerated among Black and Hispanic students 
since thgy are mere likely to be overaged ( or undergraded) Ortiz ( I 986). 

^ Orshansky index was omitted from regression equations at grate 8 and 1 1 s 
however f*i was significant at grade 4 and retained in the regression analyses. At grede 4, 
there may be a higher degree of unreliability in students 'self report of parental education 
and possessions in the home, and thus, the additional Mure of SES (although on a 
district- l«vel- basis) contributes in the overall equation. 

10 An examination of missing data for each race/ethnic pup due to listwisa deletion is 
included irra Appendix I, The school-level variable that eohstotly had a substantial number 
of missinfta information was remedial program. For Hispanfo students in grate 4 and 8 
where seen : pie sizes ere smaller compered to other groups, ^deletion of eases might distort 
the sample, an additional regression analysis was conducted using pairwise deletion. Results 
did not difffc^er substantially from those obtained using the listwlse deletion method, 

1 'Un&tenderdlzed regression coefficients shown in the table sr^e the average of the MOT 
effect obtarlned from regressions using each of the 5 M ptaibte " values as a dependent 
measure, T hese values are random draws from a probability distill button which estimates 
proficiencw given an individual's response to NAEP exercises end background variables. The 
spread of thro plausible values reflects the uncertainty abou* the proficiency value 
associated ^with a respondent The standard errors have been adjusted to reflect the square 
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roots of the sum of the variant duo to (mputatlcHl end the variance due to sampling which also 
includes a design effect of 2, In addition, the appropriate degress of freedom for NAEP 
analyses are based, not on the number of observations, but on the number of psus minus the 
number of strata. Thus, significance tests based on degrees of freedom on the number of 
observations will be too liberal For this reason, significance levels are not reported and 
greater emphasis is placed on the magnitude of the differences. See NAEP , User's 8uide, 1 986 
for a detailed explanation of these and other procedures to be considered when analyzing NAEP 
data 

12 In the regression equation for Black students, remedial program and per pupil 
instructional dollars acted as suppressor variables, A specific remedial program in the 
school although not correlated with proficiency (r= 01) is correlated with having a MOT 
program (r=.70) and thus adds Irrelevant variance to the variable MOT and reduces the 
relationship with proficiency. Similarly, instructional dollars per pupil although not 
correlated with proficiency (r=.G4) is correlated with MOT (r=.27) and acts in a similar 
manner. The inclusion of these two variables in the equation suppresses the unwanted 
variance in reading proficiency and incraffies the relationship between proficiency and MOT, 
For a discussion of suppression in complex regression models, see Cohen & Cohen ( i 983), 

13 The same procedures were followed as indicated in Footnote 1 1 . 
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TABLE 1 

UNWEIGHTED AND ADJUSTED WEIGHTED FREQUENCIES 
OF SCHOOLS AND STUDENTS 
BY SCHOOL RESPONSE TO NAEP MCT ITEM 

SCHOOLS . .. STUDENTS 

UHWEI6HTEP JlffilfiHIHI £ umweibhted MffiUHUft _£ 

GRADE 4 
Minimum Competency 



YES 


169 


334 


25.3 


7,226 


1 1,121 


29.0 


NO 


154 


312 


23.6 


5,289 


7,952 


20.7 


No response 


540 


676 


an 


13.489 


19.222 


50.3 


TOTAL 


663 


1,322 


100 


26,004 


38,295 


100 



Grade 8 
ninimum Competency 

YES 141 262 

NO I 1 0 235 

NO RESPONSE 235 472 

TOTAL 486 969 



27.0 6,744 13,426 32.1 

24.3 4,521 8,431 20.1 

MO. 10-575 20.004 4Zfi 

100 21,838 41,921 100 



Grade 1 1 



Minimum Competency 



YES 


1 18 


203 


30.6 


9,170 


17,621 


41.5 


NO 


82 


152 


25.2 


5,454 


9,778 


23.0 


No response 


131 


258 


44.2 


8.119 


15.067 


35,5 


TOTAL 


331 


638 


100 


22,788 


42,466 


100 
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TABLE 2 

ADJUSTED WEIGHTED FREQUENCIES OF SCHOOLS 
AND STUDENTS BY TYPE OF STATE PROGRAM 



SCHOOLS STUDENTS 





kin 




NO. 


% 


Grade 4 










MOT ^yes/State controlled 


72 


13.9 


2,598 


1 9.2 


MCT-yes/Optfons 


138 


26.7 


3,939 


29.1 


MGT^yes/Test Program 


34 


6.6 


959 


7.2 


MCT- no/Options 


169 


32.8 


3,659 


27.1 


HOT -no/Test Program 


102 


1 9.8 


2J63 


17.4 




515 


100. 


13,518 


1 00 


Grade 6 










MCT -yes/State controlled 


58 


13.9 


2,658 


15.4 


MCT -yes/Options 


123 


29.5 


5,576 


32.3 


MCT -yes/Test Program 


31 


7.4 


448 


2.6 


MGT^no/Optionf 


\m 


35.0 


5,714 


33.1 


MCT^no/Test Program 


58 


13.9 


2,865 


16.6 




416 


100 


17,261 


100 


Grade 1 1 










MCT -yes/State controlled 


120 


34.8 


7,650 


35.7 


MCT-yes/Options 


59 


17.1 


4,637 


21.6 


MCT ^yes/Test Program 


8 


2.3 


293 


1.3 


MCT^no/Opttons 


107 


31.1 


5,836 


27.2 


MCT-no/Test Program 


50 


14.5 


3.037 


14.2 




350 


100 


21 ,453 


100 
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NAEP 1963=84 READING GRADE 4/ AGE 9 
Selected Demographic and School Characteristics 



PERCENTAGE OF SCHOOL RESPONSES 



Minimum Competency 



Yes 



No 



No 

Response 



Northeast 


24,6 


22.8 




23,3 


Southeast 


22,4 


9.7 




23,7 


Central 


23.8 


45.7 




29.8 


T t Eg I 


29.2 


21.8 








100 


100 




100 


Size & Tvoe of Community 










Rural 


1 1.9 


10.5 




8.9 


Disadvantaged Urban 


18.2 


6.4 




5.6 


Advantaged Urban 


1 1.4 


9.3 




13.9 


Big Cities 


8.4 


5.8 




8,4 


Fringe of big cities 


16.4 


9.7 




5.8 


Medium Cities 


14.0 


1 1.3 




19,4 


Small places 


19.8 


47.0 




38.1 




100 


100 




100 




WEIGHTED SCHOOL MEANS 




Minimum competency 


NO 






Yes 


No 


Response 


School Composition & SES 










S White students 


72 


87 


83 




S Black students 


19 


7 


1 1 




S Hispanic students 


9 


6 


6 




Orshansky S 


1 1.3 


9.7 


1 1.4 




X of students/free lunch 


40.0 


44.0 


39.0 




Inst, $ per pupil 9 


58.9 


53.2 


56.1 




Instructional 










No, of students 


410 


338 


351 




teacher -student ratio 


1-20 


1-20 


1 -20 




* days in school yr 


178 


182 


179 




Minutes/long par class 


47 


47 


45 




■^standard class periods/day 


6.7 


7,0 


7,4 




*clas5 periods per student 


6.0 


6,4 


7.6 




S with remedial rasd.program 


57.3 


20.6 


22.1 





a aggregated from student^ level data 
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TABLE 4 

NAEP 1983-84 READING GRADE 8/ AGE 13 
Selected Demographic and School Characteristics 



PERCENTAGE OF RESPONSES 



Minimum 


Competency 


No 




Yes 


No 


Response 


Wm fl UTi null 1 1 1 t * li ■ l W\ n illii Jl 












1 °i, 1 


f 






1 1,3 


226 


CBntral 


23,9 




29.2 


Wast 


.29.0 




23.5 




100 


100 


100 


Size & Tvdc of Community 








Rural 


10.7 


17.3 


7.4 


Disadvantaged Urban 


15.0 


5.4 


8.1 


Advantaged Urban 


12.1 


9.0 


9.0 


Big Cities 


8.2 


6.9 


8.3 


Fringe of big cltira 


9.8 


12.7 


8.4 


Medium Cities 


19.3 


9.8 


12.7 


Small places 


25.0 


38.9 


46.2 




100 


100 


100 



WEIGHTED SCHOOL MEANS 
Minimum Competency No 





Yes 


No 


Response 


School Composition & SES 








% White students 


72 


86 


87 


% Black students 


17 


8 


9 


% Hispanic students 


1 1 


6 


4 


Qrshansky 2 


12.2 


9,9 


10.5 


% of students/free lunch 


38 


41 


35 


Instruction $ per pupil a 


58.9 


53,1 


56.1 


Instructional 








No. of students 


All 


437 


454 


*teacher-student ratio 


1-20 


1-17 


1-20 


# days In school yr 


179 


178 


179 


Minutes/long per class 


47 


47 


47 


■**standard class periods/day 


7.1 


7.1 


7.3 


■*cl855 periods per student 


6.2 


6,4 


6.3 


% with remedial readprogram 


75.2 


24.0 


52.3 



a aggregated from student^ level data 
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TABLE 5 

NAEP 1983-84 READING GRADE II/* AGE 1? 
Selected Demographic and School Char-aCteNstics 



PERCENTAGE OF RESPONSES 





Minimum C 


omoetency 


No 




Yes 


Nfl 


Response 


DemoqraDhic Reqion 








Northeast 


22.9 


18.0 


247 


Southeast 


31.1 


8.7 


21.6 


Central 


10.5 


43.9 


33.4 


Wast 


35.5 


29.3 






100 


100 


100 


Size & Tvoe of Community 








Rural 


1 1.1 


15.2 


10.6 


Disadvantaged Urban 


7.2 


2.0 


5.5 


Advantaged Urban 


13.4 


5.5 


15.2 


Big Cities 


9.3 


3.1 


5.5 


Fringe of big cities 


9.5 


1.4 


4.1 


Medium Cities 


15.7 


1 1.5 


13.5 


Small places 




6L4 






100 


100 


100 



WEIGHTED SCHOOL ME^ NS 



Minimum Competency No 





Yes 


No 


_BtsM 


School Composition & SES 








% White students 


77 


91 


88 


% Black students 


14 


6 


7 


% Hispanic students 


9 


3 


5 


Orshansky % 


13.7 


13.3 


1 1.6 


1 of students/free lunch 


23.8 


32.4 


26.7 


Inst $ per pupil 3 


58.5 


56.7 


58.1 


instructional 








No, of students 


960 


640 


883 


* teacher - student ratio 


1-14 


1-14 


1-1 1 


* days in school yr 


179 


179 


1 78 


Minutes/long per cites 


51 


49 


50 


-^standard class periods/day 


6.3 


6.8 


6,7 


^class period per student 


5.9 


5.9 


5,6 


% with remedial read, program 


75.1 


1 1.6 


52.3 



aggregated from student- level data 
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TABLE 6 
GRADE 4 

SCHOOL AND STUDENT-LEVEL CHARACTERISTICS 
WEIGHTED AVERAGES" BY RACE/ETHNIC GROUP BY SCHOOL TYPE 



WHITE BLACK HISPANIC 





r 




MCI 


MCT 










Yes 


No 


Yes 


Np. 


School level Variables 

% White students 


77 
(21) 


91 

(12) 


35 
(30) 


37 
(38) 


49 

(27) 


62 
(36) 


Orshansky % 


12 

(11) 


8 

(8) 


17 
(10) 


19 
(13) 


15 
( 13) 


13 
(13) 


% of students/free lunch 


39 
(32) 


39 
(34) 


64 
(32) 


62 
(30) 


46 

(33) 


48 

(37) 


Instructional $ per pupil 


58 
(23) 


49 
(23) 


58 
(22) 


63 
(15) 


60 
(22) 


50 
(28) 


student- level variables 














Parents' Education 


3.03 
( 1.10) 


3.09 
(1.15) 


3.02 
( 1.11) 


3.06 
(1.04) 


2.93 2.79 
( 1.15X1.04) 


Family Background (b) 


6.07 
(2.14) 


6.21 
(2.02) 


5.71 
(2.17) 


5.82 
(2.10) 


5.42 5.47 
(2.28) (2. 11) 


Students Academic 
Behaviors 


3.44 
( 170) 


3.34 
(1.67) 


3.46 
( 1.75) 


3.26 
( 1.80) 


3.36 3.21 
( 1.62) (1.61) 


Students' Age 


9.25 
(.52) 


9.29 
(.51) 


9.34 
(.61) 


9.45 
(.67) 


9.38 
(.65) 


9.36 
(.61) 



a Standard deviations in parentheses 

b Composite of parents' education plus possessions in the home. 
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TABLE 7 
GRADE 8 

SCHOOL AND STUDENT LEVEL CHARACTERISTICS 
WEIGHTED AVERAGES" BY RACE/ETHNIC GROUP BY SCHOOL TYPE 



WHITE BLACK HISPANIC 





MCT 


MCT 


MCT 




Yes 


No 


Yes 


No 


Yes 


No 


School-Level Variables 














X white students 


76 

(22) 


89 
( 16) 


37 
(31) 


43 
(34) 


39 
(27) 


39 
(26) 


Orshansky % 


10 

(9) 


I | 

(10) 


1 7 
(10) 


23 
(25) 


1 1 

(7) 


27 
(18) 


% of students/free lunch 


23 
(24) 


41 

(30) 


49 
(28) 


58 
(33) 


44 
(27) 


6! 
(30) 


Instructional $ per pupil 


63 
(IS) 


53 
(13) 


65 
(16) 


55 
(11) 


66 
(14) 


48 
(11) 


Student- Level Variables 














Parents' Education 


3.01 
(1.03) 


2.71 
(1.02) 


2.72 
( 1.06) 


2.62 
(1.10) 


2.52 2.05 
£ 1.14) ( 1.09) 


Family Background (b) 


7.23 
(1.62) 


7.03 
(157) 


6.39 
(1.87) 


6.27 
(1.95) 


5.70 5.24 
(2.03) (2.04) 


Students' Academic 
Behaviors 


3.63 
(1.90) 


3.58 
(1.68) 


3.53 
(1.75) 


3.48 
(1.70) 


3.56 3.33 
( 1.77) £ 1,70) 


Students' Age 


13.3 
(.52) 


13.3 
(.53) 


13.4 
(.73) 


13.7 
(.90) 


13.4 
(.57) 


13.6 
(.70) 



a Standard deviations in parenthesis 

b Composite of parents 4 education plus poisesslons in the home. 
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TABLE 8 

SCHOOL AND STUDENT-LEVEL CHARACTERISTICS 
WEIGHTED AVERAGES* BY RACE/ETHNIC GROUP BY SCHOOL TYPE 



Grade 1 1 

WHITE BLACK HISPANIC 



MCT MCT MCT 

Yes No Yes No Yes No 



Sc hool -Level Variables 



1 White students 


79 


88 


45 


62 


35 


55 




(19) 


(15) 


(32) 


(21) 


(29) 


(31) 


Orshansky % 


12 


12 


19 


32 


22 


21 


(11) 


(13) 


(12) 


(21) 


(17) 


(12) 


% of students/free lunch 


15 


22 


38 


39 


34 


40 




(16) 


(20) 


(27) 


(25) 


(23) 


(25) 


Instructional $ per pupil 


61 


57 


65 


50 


62 


59 


(17) 


(14) 


(15) 


(19) 


(16) 


(11) 



Student Level Variables 



Parents' Education 


3.08 2.74 


2.61 


2.33 


2.14 2.04 




( 1.02) (1.01) 


(1.09) 


(1.00) 


( 1.14) ( 1.00) 


Family Background (b) 


7.60 7.25 


6.72 


6.40 


5.91 5.83 


( 1.47) ( 1.41) 


(1.66) 


(1.62) 


(1.87) (1.71) 


Students' Academic 


4.30 3,93 


4.12 


3.86 


3.93 3.72 


Behaviors 


(2.06) (1.94) 


(1.91) 


( 1.94) 


( 1.93) ( 1.93) 


Students' Age 


17.0 17.0 


1 7.3 


17.3 


17.3 17.4 


(.60) (.50) 


(.80) 


(.80) 


(.80) (.70) 



3 Standard deviations in parentheses 

b Compoiiti of parents* education plus possessions In the home. 
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TABLE 9 

UNADJUSTED AVERAGE READING PROFICIENCY BY GRADE BY 
RACE/ETHNIC GROUP BY SCHOOL TYPE 8 

GRADE 4 GRADE 8 GRADE 1 1 

HCT No MCT No OCT No 

Yaa No Response Yei _No Response Vis No Response 

White 222.0 227.0 219.6 267.7 260.6 266.9 298.1 290.8 291.1 
(31.5) (30.5) (32.73 (27.2) (28.5) (27,7) (29.9) (31.5) (32.3) 

Black 194.9 194.4 189.0 243.9 232,3 239.5 267.7 262.7 261.6 

(28.3) (286) (30.5) (27.2) (27.8) (28.0) (26.2) (31.5) (31.3) 

Hispanic 197.9 202.3 194.9 243,7 236.4 244.1 269.0 261.9 263.8 
(31.1) (29.8) (32.7) (29.8) (28.5) (27.2) (32.2) (34.4) (34.0) 



a Standard deviations In parentheses. 
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TABLE 10 

UNSTANDARDIZED REGRESSION WEIGHTS FOR 
MINIMUM COMPETENCY DUMMY-CODED VARIABLE 





GRADE 4 




GRADE 8 


GRADE 


1 1 






Final b 




Final 




Final 




Adjusted 8 


Step 


Adjusted 


Step 


Adjusted 


Step 


White 


-1.66 


-1.08 


7.79 


5.54 


2.18 


.42 


(S.E.) 


(1.09) 


(1.08) 


(1.29) 


(1.51) 


(1.01) 


(1.58 


Black 


.30 


2.89 


10.90 


7.60 


6.62 


12.34 


(S.E.) 


(2.58) 


(3.97) 


(2.51) 


(3.23) 


(2.91) 


(3.48; 


Hispanic 


-2.86 


-3.00 


-1.2! 


.06 


5.93 


5.76 



(S.E.) 



(3.01) (3.23) (330) (3.71) (3,48) (433) 



a at each grade level, effect adjusted for sex, student age, region of the country, school-level 
SES, family background, and student academic behaviors, 

^ at each grade level , effect adjusted for student and school level variables in addition to 
school-level remedial program (dummy coded 1 -ym, 0=no) and instructional dollars per 
pupil 
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TABLE S 1 

UNADJUSTED READING PROFICIENCY BY SCHOOL RESPONSE 
PROGRAM BY TYPE OF STATE MANDATE 

Grade 4 

TYPE OF STATE MANDATE 
MOT -state controlled MCT- local options Teat Program only 

NAEP Response 

MCT-YES 210.4(32.6) 2149 (33.3) 213.8 (.5.5.4) 

MCT-NO * 221.3 (31.7) 224.8 (32.1) 



Grade 8 

TYPE OF STATE MANDATE 
MCT -state controlled MCT- local nations Test Program only 

NAEP Response 

MCT-YES 258.9 (29.0) 260.9 (29.6) 267.3 (30.6) 



MCT-NO * 258.9(29.0) 251.0(32.0) 



Grade 1 I 

TYPE OF STATE MANDATE 
MCT-state controlled MCT- local options Test Program jnLY 

NAEP Response 

MCT-YES 289.7(32.7) 287.4(32.9) 269 ! (34.2) 



MCT-NO « 

•omitted from all analyses. 



288.5 (32.8) 
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TABLE 12 

UN5TANDARDIZED REGRESSION WEIGHTS BY TYPE OF STATE 
MANDATE FOR MINIMUM COMPETENCY TESTING 3 

GRADE 4 GRADE 8 GRADE 1 1 

B 5E B SE B SE 

Sch. Yes MCT/State-controlled -3.89 (1.15) 9.19 (.49) 7.10 (1.42) 

Sch. Yes MCT/SLite-optlon(s) -1.81 (1.41) 8.28 (.79) 5.63 (1.24) 

Sch. Yes riCT/state test -3.59 (2.22) 427 (240) -.29 (403) 

Sch. No MCT/State-optionCs) -1.78 (1.18) 3.56 (.86) 1.56 (1.49) 

(Rer. Group Sch. No MCT7 
Slate-no MCT) 



a Adjusted for district and school context and SES, parents' academic values, and race. 
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Figure 1 

fiveragre Reading Proficiency by race/ethnic gfroup 
by grade level for total NAEP Sample 




Figure 2 

Proportion of NAEP 4th Grade White Students at or Below 
Each Proficiency Level by MCT School Response 




Fl yur t 3 

Proport i on of NAEP 4th Grade Black Students at or Below 
Each Pr of i c i ency Levi! by MCT School Response 




Fi grure 4 

Proportion of NAEP 4th Grade Hispanic Students^ at or Below 
Each Prof i c i ency Level by MCT School Response 



bin 




Rudiment. Bas i c In termed. Adept Advanced 

NAEP Readingr Proficiency Levels 
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Proportion of NAEP 8th Grade White Students at or Below 

Each Proficiency Leve 1 




Figrure b 

Pr oq port i on of NAEP 8th Grade Black Students at or Below 
Each Proficiency Level by MCT School Response 




J&iosic Intermediate Adept Advanced 

NfiEP Readiny Proficiency Levels 
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Figure 7 

Proportion ofNftEP flfch Grade Hispanic Students at or Be t omj 
Each Prof i c i era ey Level by MCT School Response 

b8 n 




Figure B 

■tjMof NAB£P 11th Grade White Students at or Below 
gach Prpf i c i ency Level by MCT School Response 




I- ntermed. Adept 
NAEP Reading? Proficiency Levels 



NO RESPONSE 
MCT- NO 
MCT- YES 



Advanced 
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Fijure 9 

Proportion of NAEP 11th Grade Black Students at or Below 
Each Proficiency Level by MCT School Response 

70-i 




Bas i c Intermed. Adept Advanced 

NAEP Riadingr Proficiency Levels 



Fl yuT — e IB 

Proportion of NftEP 4th Grade Hispanic Students at or Below 
Each Prof i c i «ntj Leva I by MCT School Response 



b8 — , 




Basic lr\itYmi Adept Advanced 

NABP R«4difi» Pro^fi c i ancy Levels 
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APPENDIX A 
NAEP PROFICIENCY LEVELS 

Rudimentary 

Performance at the rudimentary level suggests that r^*ders have acquired the ability to 
carry out simple, discrete reading tasks; e.g., can foliov^v brief written directions, select 
words, phrases or sentences to describe a picture, PerfcDrmanra at the basic level suggest 
the readers' ability to understand specific or sequentially r— elated information; e.g., can locate 
and identify facts from simple informational paragraphs, ^stories, and can combine ideas and 
make inferences based on short, simple passages, 
Basic 

Performance at the basic level supts the readers have I^earnad basic comprehension skills 
and strategies and can locate and identify facts from s- imple Informational paragraphs, 
stories, and news articles, in addition, they can combine i : deas and make inferences based on 
specific or sequentially related information. 
Intermediate 

Performance at the intermediate level suggests the readers 1 ability to search for specific 
information, Interrelate ideas, and make generalizations. , These readers can search for, 
locaet, and organize the information they find In lengthy passages and can recognize 
paraphrases of what they have read. They can also make inferences and arrive at 
generalizations about main ideas and author's purpose fro^m passages concerning literature, 
science, dnd social studies. 
Adept 

Performance at the adept level suggests the readers ability^- to find, understand, summarize, 
and explain relatively complicated information. For e^am jple, these readers can understand 
complicated literary and informational passages, and can amalyze and integrate less familiar 
material 
Advanced 

Performance at the advanced leval suggests the ability— to synthesize and learn from 
specialized reading materials. For example these readers can extend and restructure the 
ideas presented in such texts as literary essays, scientific rr — iaterials, historical and technical 
documents. 



Taken from NAEP. Reading Report Card. 1 985, 



9.5 
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APPENDIX B 

CODING OF TYPES OF STATE PROGRAMS 

The basis for categorizing state-mandated programs derived initially from obtaining a 
consensus of information about characteristics of each state progr^sm. Information obtained 
from published listings, ERIC reports, and published works on nijlnlmum competency testing. 
Published listings provided information on the status of state r^lCT programs, and typically 
listed options available to school districts. Options might include - which grade level to test, the 
actual test used, or substitution of a local assessment progr&sm in lieu of a state testing 
program. For purposes of categorization, no distinctions were macde between types of options. 

However, on any one listing, two very different state ncT ^programs might appear to be 
similar when in reality they are very different. For example, an one listing, both Delaware 
and Florida are listed as hayings state MOT with the state apwicy setting the performance 
standards, and attainment of standards required for high school g^^aduation. In Delaware, the 
state educational agency provides the specific competencies to be achieved, and the local 
districts determine in what manner these competencies are to be assessed. Completion of 
state^wide competencies are required for high school graduation, standardized test , however, 
is not necessarily a part of this program, There is in Delaware a separate and distinct 
state-wide assessment program, In Florida, the state educational agency develops minimum 
student performance standards, testing Instruments, and attests those standards for pupil 
promotion and graduation, Theseare twovery different state Prpffiirams, 

Different types of MCT programs which appear similar on pwublishsd listings may result 
from the confusion between MCT and competency- based education, two different type of 
programs. Competency based Mucation often includes minimum ccDmpetency tests as one of its 
components, CBE attempts to set clearly defined objectives for ail levels of education and has a 
strong focus on basic skills. Thegoals In CBE are generally mofte ambitious and less minimal 
than those specified in MCT, Hower, in many districts that uts c^ompstency -based education, 
students must also pass a competency exam before graduation. Otheer districts require students 
to master a minimal core of competency based objectives (lajarmis, 1981 ), Differences on 
published listings might also result from other terminology used by state officials to describe 
these programs, Thus, published listings may be adequate for surwrnarlzing the various state 
programs; however, may reflect similarities between progranms that are, in fact, quite 
different, Others have also identified discrepancies between various published listings 
(Marshall, 1986), For purposes of cat^orization in this stutf^-y, no distinction was made 
between com patency- based education andminimum competency testing. The working definition 
of MCT, used in classifying programs was the one used by Educatioten Commission of the State 
In general, MCT programs are three in which (a) there is g statos mandate to test all students 
in one or more grade and (b) theri Is an effort to set predeterminscd minimum standards either 
statewide or locally. 

The time period in which the lists and reports were publi^shed was also considered in 
the classification of a state- MCT program. For purposes of eddying, the legislation or board 
mandate must have been in effect by April-May, 1984 (during the time of the NAEP data 
collation). Thus, those states which implemented a MCT program after this date would be 
included in one of the two classifications of state testing programs. 
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APPENDIX B-2 
MINIMUM COMPETENCY TESTING PROGRAMS BY STATE 

STATE-MANQATPn MCT PROGRAMS- STATE CONT ROLLED 

Govn.Level 
Action Setting 
State Taken Standard Element ary Middle High Schoo 



Alabama 

Arizona 1 

Arkansas 2 

Florida 

Georgia 

Hawai 1 3 

Kentucky 

Kansas 4 

Louisiana 5 

Maryland 

Mississippi 6 

Missouri 

Nevada 

New Jersey 

New York 

South Carol. 7 

Tennessee 

Texas 8 

Virginia 

Vermont 

Rhode Island 

North Carolina 

Pennsylvania 9 



1977 SEA/SBE 
1976,83 SBE/LEA 
1 979 5BE 
1976,83 SBE/LEA 
1976,81 SBE /SEA 

1978 SEA 
1977,78 SEA 

1978.81 SEA/SBE 
1976,77 SEA/SBE 

1976.77 SBE/SEA 
1 982 SEA 

1976.78 SEA 

1977.79 SBE 
1976,79 SBE/LEA 

1978 SBE 
1978,84 SBE 

1977.82 SBE/LEA 

1 979 SEA 
1976,78 SEA/LEA 

1 977 SBE 

1978 5BE/5E/ 
1977 SBE 

1 984 SEA 



x 
X 
x 
x 

x 

X 
X 
X 
X 
X 

X 

X 
X 
X 
X 
X 
X 
X 
X 
X 

V* 

X 



X 
X 

X 

X 
X 
X 



X 
X 

X 
X 



X 
X 
X 



graduation requir. 
graduation requir. 

graduation requir, 
graduation requir. 
graduation requir. 



graduation requir. 
graduation requir. 
graduation requir. 

graduation requir. 
graduation requir. 
graduation requir. 
graduation requir. 
graduation requir. 
graduation requir. 
graduation requir. 
graduation requir. 

graduation requir. 



|| ATP MINIMU M rnMPETFNCV TFSTIMB WITH I PC Al OPTION(S) 

~" ~~ x graduation requir. 

graduation requir. 
graduation requir. 
graduation requir. 

x 
x 



California 


1 976,83 


LEA 


X 


Colorado 


1975 


LEA 


X 


Delaware 


1976 


SEA/LEA 




Idaho 


1977 


SBE/SEA 


X 


Connecticut 


1978 


SBE/LEA 


X 


Indiana 


1978 


LEA 


X 


Massachusetts 


1978 


LEA 
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APPENDIX B-3 



Govn.Level 





Action 


Setting 




State 


Taken 


eitanHarrt Fifimentarv Middle 


New Mexico 


1976 


5BE 




New Hampshire 1977 


SEA/LEA 


x x 


Nebraska 


1975 


SEA/LEA 


X 


Illinois 


1978 


LEA 


At local option 


Ohio 


1982 


LEA 


At local option 


Oregon 
Utah 


1976,80 


SEA/LEA 


At local option 


1977 


SEA/LEA 


At local option 


Wisonsin 


1981,82 


LEA 


At local option 


Wyoming 


1977,80 


SEA/LEA 


At local option 



Hi gh School 



graduation requir. 
graduation requir. 



ill STATE TFSTINO PROGRAM ONLY 



Alaska 
Maine 
Minnesota 
Michigan 
Oklahoma 
South Dakota 
Washington 
West Virginia 

I V NO STATF TESTING PROGRAM 



Iowa 
Montana 
North Dakota 



Abbreviations and Symbols 
LE8 - State Legislature 
8BE - State Board of Education 
SEA - State Education Agency 

LEA - Local Education Agencies (local school boards or school districts) 
"x" indicates that a state wide MCT mandate exists at elementary (grades 1 through 6), middle 
(grades 7-8) or high school (grades 9 through 1 2). 

-graduation requirement" listed under high school indicates that some ^rm of exit or competency 
exan i is required prior to graduation. The exam could occur in either of grades 9 , 10, 1 1 or 12, 
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APPENDIX B-4 

NOTES to Table; 

' Legislation in 1983 calls for Arizona to develop o minimum course of study and criteria for 
high school graduation standards and for grade to grade promotion criteria. Local school 
districts are to implement standards. 

2 | n ,987,8 minimum comptency test will be administered for 8th grade promotion. 

3 For high school graduation requirements students have 3 options: paper-pencil test; 
performance test; or course. First time taken (grade 9) must be paper-pencil. 

« The Kansas Minimum Competency Assessment (MCA) was reestablished by 1984 legislative 
action (SB. 473). The MCA will be in effect 1984-85 through 1988-89. 

5 Louisiana will add 8th grade beginning with 1986-87 school year and will implement a 
graduation requirement, 

6 State Board developed 1 1th grade functional literacy test necessary for graduation (December, 
1982, effective, 1986). 

7 The South Carolina Education Improvement Act of 1 984 specifies that the 1 1 th grade test being 

used to gather baseline data will bereplacedin the 1 985-86 school year with an exit worn in 
the 1 0th grads. All students graduating in 1990 or after must pass the exam (June lytHi. 

8 Texas-High school students must pass exit exam (July 1984, effective June 1986). 

9 Will require reading and math test in grades 3,5, and 8; state -funded remedial program for 
those who fail state-wide test, June, 1 984. 

Information comnilad in this lintimi was obtain wt from the following sourees: 

The Notion Responds- Recent Fffnrts to Imp rove Education. U. S. Dept of Education, u. 5. 
Sovernment Printing Office, Washington, D. C. May,! 984. 

Pi isko Y and J D. Stern (Eds.) The Condition of Education. 1985 Edition, P. 68, Table 1.26 
"States Using Minimum-Competency Testing, by Government Level Sitting Standards, Grade 
Levels Assessed, and Expected Use of Standards. United States. U.S. 0ovn. Printing Office. 

Education Week, "Changing Course A 50-state survey of reform measures. February 6, 1985, 
Pps. 11-30. 

Sutter. J. A. & Rice, E. V. sturtant Performance Standards and Tilin g Program* Baekgrouo d 
information for legislators, Minnesota Senate, St Paul. Senate Counsel and Research, March 
1 984 Eric Document Reproduction Service 252 530, 

Plpho, Chris. State Activity; Minimal Comp etency Testing. Denver, Colorado; Education 
Commission of the States, November , 1981 

ECS Issuegram; student Minimum H nmnetencv Testing. Denver , Colorado; Education 

Commission of the States , January , 1 983, 

Ooertz, M. E. State Educational Standards: A fi n-State Survey. Educational Testing Service, 
September, 1985. 
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APPENDIX C 

DESCRIPTION OF NAEP SIZE & TYPE OF COMMUNITY VARIABLE 

The size and type of community variable provides information about the communities In 
which the schools were located. The categories consisted of three "extreme" types of 
communities e.g., rural, disadvantaged urban and advantaged urban, and four "residual 
community types based on population size. Schools include in the extreme rural, and 
disadvantaged or advantaged urban areas consist of the top 10 percent of schools In a 
rank-ordered listing of occupational profile of residents, size of the population, and type of 
community. The remaining schools were classified according to one of the four residual 
categories - main big city, urban fringe, medium city, and small place depending upon the size 
of the community in which they were located ( NAEP Users' Quids, 1 985). 

f ytreme Rural 

This category was used for schools in rural areas where a high proportion of adults were 
farmers or farm workers and a low proportion were professional .managerial, or factory 
workers. At least some of the students in these schools were from open country or plows with 
a population of less than 1 0,000. 

Disadvantaged Urban 

This category was used for schools in areas where a high proportion of the adult 
population was either not regularly employed or on welfare and a low proportion was 
employed in professional or managerial positions. The schools in this category were located ,n 
cities, or the urbanized area of cities, with a population greater than 200 ,000. 

Advantaged Urban 

This category was used for schools in areas where a high proportion of adults were 
empleysd in professional or managerial positions and a low proportion of factory or farm 
workers not regularly employed, or on welfare. The schools in this category were located In 
cities or the urbanized area of cities with populations greater than 200 ,000. 
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APPENDIX C-2 

DESCRIPTION OF NAEP SIZE AND TYPE OF COMMUNITY VARIABLES 



Main Big City 

This category was used for schools located within the limits of cities with populations 
greater than 200,000 but not classified as advantaged or disadvantagsd urban. 

Urban Fringe 

This category was used for schools located in an urbaniied area, but outside the limits of 
cities with populations over 200 ,000 and not classified as advantaged or disadvantaged urban. 

Medium City 

This category was used for school located in cities with populations of between 25,000 
and 200 ,000 which did not classify as fringe areas for big cities. 

Small Place 

This category was used for schools located in communities with populations of less than 
25,000. These communities were not located in an urbanized area of big cities and could not 
be classified as "Extreme Rural". 



Taken from mafp User'a Guide . 1 985, 
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APPENDIX D 



srHOOL AND STUDENT-LEVEL CHARACTER 1ST ICS 
WEIGHTED I AVERAGES BY RACE/ETHNIC GROUP 
BY GRADE FDR NON-RESPONSE SCHOOLS 



WHITE 



Mean * 



GRADE 4 

Schoul-lavel Variables 

% White students 85 

Orshansky % 1 2 

% of students/free lunch 36 
Instructional! per pupil 5k 

Student- level variables 

Parents' Education 3 0. 

Family Background a 6.17 
Students' Academic Behaviors 3.38 
Students' Age 9 28 

GRADE 8 

School -leva! Variables 

% White students 

Orshansky % 
% of students/free lunch 
Instructional $ per pupil 

Student- level variables 
Parents' Education 2,77 
Family Background (a) 7, 1 .7 
Students' Academic Behaviors 3.59 
Students' Age 13.3 



(1.05) 
(1.59) 
(1.68) 
(.62) 



HISPANIC 



— . — 

? j. 




S.D, 


Mean 


3D. 




47 


(38) 




\o f l 




2Q 


(15) 


21 


(20) 


'31) 


f 0 


(32) 


49 


(35) 


a.) 


55 


(24) 


53 


(22) 


V 


2.86 


(1.12) 


2.78 


(1.17) 


(2(1?) 


5.55 


(2.05) 


5.47 


(2.13) 


(1.66) 


3.34 


(Lit) 


3.40 


(1.69) 


( .50) 


9.41 


(.64) 


9.36 


(.62) 



86 


(17) 


55 


(34) 


54 


(38) 


1 


(12) 


21 


(15) 


17 


(14) 


32 


(29) 


49 


(30) 


50 


(31) 


54 


(16) 


60 


(16) 


54 


(15) 



2.58 
6.26 
3.48 
13.7 



(1.07) 
(1.81) 
(1.71) 
(.90) 



2.28 (1.15) 
5.50 (2.17) 

3.29 (1.72) 
13.6 (.76) 
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APPENDIX D-2 

SCHOOL AND STUDENT-LEVEL CHARACTERISTICS 
WEIGHTED AVERAGES BY RACE/ ETHNIC GROUP 
BY GRADE FOR NON-RESPONSE SCHOOLS 



WHITE 



Mean S.D. 



GRADE 1 1 

School -level Variables 

1 White students 
Orshansky % 
% of students/ free lunch 
Instructional $ per pupil 
Student- level variables 
Parents' Education 
Family Background (a) 
Students' Academic Behaviors 4, 1 2 (1.91) 
Students' Age 17.1 (.50) 



2.76 (1.05) 
7.37 (1.39) 



BLACK 



Mean S.D. 



2.57 
6.67 
4.12 
17.3 



(1.07) 
(1.67) 
(1.93) 



(.80) 



HIS PANIC 
Mean S.D. 



92 


(14) 


77 


(30) 


51 


(40) 


10 


(9) 


17 


( H) 


21 


(20) 


18 


(22) 


44 


(38) 


48 


(39) 


58 


(13) 


63 


(14) 


61 


(12) 



2.19 (1.20) 

5.91 (2.01) 

3.94 (1.98) 

17.3 (.80) 



a Composite of parents' education plus possessions in the home. 
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APPENDIX E 



UNADJUSTED READING PROFICIENCY 
BY GRADE BY RACE BY SCHOOL TYPE 
BY REMEDIAL PROGRAM IN READING 



White Students 



GRADE 4 
REMEDIAL PROGRAM 







YES 


NO 




MCT PROGRAM 


YES 


221.8 
(31.2) 


221.5 
(31.5) 


221.7 
(31 4) 




NO 


227.2 
(30.3) 


225.8 
(30.6) 


226.2 






223.2 
(31.3) 


224.2 
(30.9) 


223.7 
(28.0) 


Black Students 


REMEDIAL PROGRAM 
YES NO 




MCT PROGRAM 


YES 


193.2 
(28.0) 


202.3 
(26.2) 


194.3 
(28.0) 




NO 


183.7 
(28.0) 


194.4 
(27.2) 


192.0 
(27.9) 


Hispanic Students 




192.7 
(28.1) 


197.5 
(27.1) 


193.8 
(27.9) 




REMEDIAL PROGRAM 
YES NO 




MCT PROGRAM 


YES 


198.9 
(31,2) 


196.8 
(30.9) 


197.8 
(31.1) 




NO 


203.2 
(26.9) 


205.4 
(32,2) 


206.2 
<50.1) 






200,7 
(30.6) 


200.0 
(31.7) 


200.3 
(31 2) 
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APPENDIX E-2 



GRADE 8 



White Students 



MCT PROGRAM 



Black Students 



MCT PROGRAM 



REMEDIAL PROGRAM 
YES NO 



YES 268.2 
(26.9) 



265.0 267.3 
(27.7) (27.2) 



NO 264.9 
(27.8) 



258.2 
(29.7) 



259.8 
(29.4) 



267.5 260.3 263.8 

(27.1) (29.3) (28.5) 

REMEDIAL PROGRAM 
YES NO 



YES 244.6 
(26.7) 



240.8 
(28.1) 



NO 



229.6 

(26.2) 



233.6 
(29.1) 



240.9 
(28.9) 



236.7 
(27.4) 



243.6 
(27.1) 

231 8 



239.4 
(28.0) 



Hispanic Students 



MCT PROGRAM 



REMEDIAL PROGRAM 
YES NO 



YES 243.5 243. 

(29.2) (28.9) 



NO 238.8 
(28.1) 



243.2 
(29.2) 



234.1 
(29.4) 



243.4 

(29.0) 

234.5 
(29.4) 



238.5 240.6 
(29.4) (29.4) 
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APPENDIX E-3 
GRADE 1 1 



White Students 



MCT PROGRAM 



Black Students 



MCT PROGRAM 



Hispanic Students 



REMEDIAL PROGRAM 
YES NO 



YES 298 6 295,3 298.1 

(29,5) (31.8) (29.9) 



NO 292,2 292.1 292.1 

(31.4) (31.2) (31.2) 



297.8 292.8 295.8 

(29.8) (31.4) (30.5) 



REMEDIAL PROGRAM 
YES NO 



YES 267.8 274.2 268.3 

(28.2) (27.2) (28.1) 



NO 260.7 263.9 262.9 

(32.6) (29.7) (30.7) 



267.2 267.2 267.2 

(2.8.6) (29.3) (28.8) 



REMEDIAL PROGRAM 
YES NO 



249.2 269.0 
(27.3) (32.4) 



NO 261.9 266.3 264.7 

(31.3) (34.0) (33.1) 



269 1 262.7 268.5 

(32.3) (33.5) (32.6) 



MCT PROGRAM YES 269.7 

(32.4) 
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APPENDIX F 

DISTRIBUTION OF NAEP PROFICIENCY LEVELS 
BY RACE/ETHNIC GROUP AND SCHOOL TYPE 

Wtightsd psrcsntags of stud§nts at or bilow levih 
Ki RUDIMENTARY BASIC INTERMEDIATE ADEPT ADVANCED 



GRADE 4 
WHITE 



MCT-YES 


4809 


.5 


26.3 


53.5 


19.4 


MOT -NO 


5366 


.9 


26.5 


51.9 


20.7 


NO- RESPONSE 


12210 


.4 


23.8 


54.1 


21.6 



BLACK 



MCT-YES 


1567 


5.0 


55.6 


37.6 


1.9 


MCT-NO 


493 


12.6 


55.4 


30.6 


1.4 


NO- RESPONSE 


2495 


6.1 


55.4 


35.2 


3.2 


SPANIC 












MCT-YES 


914 


3.4 


53.6 


35.6 


7.1 


MCT-NO 


683 


7.8 


50.4 


35.2 


6.6 


NO- RESPONSE 


1982 


4.3 


45.7 


44.6 


5.4 



GRADE 8 



WHITE 



MCT-YES 


5535 


1.0 


25.0 


63.6 


10.5 


MCT-NO 


539 


1.2 


27.9 


61.2 


9.6 


NO- RESPONSE 


12686 


1.3 


25.1 


63.9 


9.7 


,ACK 












MCT-YES 


1312 


4.7 


51.5 


42.4 


1.4 


MCT-NO 


641 


8.0 


57.2 


34.4 


.4 


NO- RESPONSE 


2394 


9.1 


53.8 


36.3 


.7 


SPANIC 












MCT-YES 


1019 


8.7 


50.2 


39.8 


1.3 


MCT-NO 


533 


8.2 


53.7 


37.7 


.4 


NO- RESPONSE 


1347 


5.8 


51.1 


42.7 


.3 
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APPENDIX F-2 

DISTRIBUTION OF NAEP PROFICIENCY LEVELS 
BY RACE/ETHNIC GROUP AND SCHOOL TYPE 



Weighted percentage of students at or below level: 





.N_ J 


BASIC 


INTERMEDIATE 


_A.Q£EL 


ADVANCED 


GRADE I 1 












WHITE 












MOT -YES 


8441 


.1 


51.1 


7.1 


41.7 


MOT -NO 


6143 


.3 


8.7 


45.4 


45.7 


NO-RESPONSE 


9829 


,2 


9.1 


43.1 


47.6 


BLACK 












MOT -YES 


2386 


.9 


26.6 


60.3 


12.2 


MOT -NO 


784 


1.1 


30.4 


55.1 


13.4 


NO- RESPONSE 


1672 


1.6 


32.9 


53.0 


12.5 


HISPANIC 












MCT-YES 


1641 


1.6 


27.1 


53.2 


18.1 


MOT -NO 


31 1 


2.6 


27.1 


53.1 


17.2 


NO- RESPONSE 


815 


1.2 


27.7 


51.7 


19.4 



Sample sizes weighted and adjusted to reflect unweighted number of cases and divided by 8 
design factor of 2. 
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APPENDIX G 

PERCENTAGE OF STUDENTS IN STUDY 2 
REGRESSION ANALYSES 
BY GRADE BY RACE/ETHNIC GROUP 



WHITE BLACK HISPANIC 

73.7 14.9 M.4 

Total NAEP sample 3 70.5 15.2 11.3 



Grade 4 

Study 2 subsample 



Grade B 

Study 2 subsample 

Total NAEP sample 

Grade 1 1 

Study 2 subsample 

Total NAEP sample 



74.7 14.3 M.O 

74.0 14.4 8.7 

74.7 15.8 9.6 

74.3 14.9 7.9 



3 percentage of Total NAEP grade sample based on all students including those in race group 
"other". 
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APPENDIX H 

ZERO ORDER CORRELATION COEFFICIENTS 
MCT DUMMY CODE AND STUDENT 
READING PROFICIENCY 



White 
Black 
Hispanic 



fiRADE 4 

-.067 

.024 
-.122 



GRADE 8 

.151 

.256 
.033 



fiRADE 1 1 

.1 12 

.068 
.092 



ERIC 
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APPENDIX I 

PERCENTAGE OF CASES FOR VARIABLES INCLUDED IN 
REGRESSION ANALYSES WITH MISSING DATA 



WHITE BLACK HISPANIC 

Grade 4 

Students' Academic Behaviors 8,8 8.0 12.0 



Family Background 
Remedial program 
% free lunch 
Instructional $/pup11 °' 
% of White students 



5 

5.8 8.1 19. 

7,4 6.8 10.6 



Grade 8 

Students' Academic Behaviors 
Family Background 
Remedial program 
% free lunch 

Instructional $/pupil 11.1 
% of White students 2.2 01 



42 2 8.7 

8.3 8.2 

4 5 .01 3.6 

mi - 6.2 



Grade 1 1 

Students' Academic Behaviors 
Family Background 
Remedial program 
% free lunch 



.9 2.4 7.0 

17.5 19.1 13.6 

7,9 10.0 4.3 



Instructional $/pupil 9.3 2.2 6.5 



% of White students 



2.7 .01 



in 



